United  States  Military  Academy 
West  Point,  New  York  10996 


OPERATIONS  RESEARCH  CENTER  OF 
EXCELLENCE  TECHNICAL  REPORT  DSE-TR-02-02 

DTIC  #:  TBD 


Colonel  Bill  Klimack,  Ph.D. 

Director,  Operations  Research  Center  of  Excellence 

Approved  by 

Colonel  Michael  L.  McGinnis,  Ph.D. 

Professor  and  Head,  Department  of  Systems  Engineering 


June  2002 


The  Operations  Research  Center  of  Excellence  is  supported  by  the 
Assistant  secretary  of  the  Army  (Financial  Management  &  Comptroller) 

Distribution  A:  Approved  for  public  release; 
distribution  is  unlimited. 


20020/30  245 


AQU02-/Q -  j3gr 


Hybrid  Value-Utility  Decision  Analysis 

Colonel  Bill  Klimack,  Ph.D. 

Director,  Operations  Research  Center  of  Excellence 


OPERATIONS  RESEARCH  CENTER  OF  EXCELLENCE 
TECHNICAL  REPORT  DSE-TR-02-02 
DTIC  #:  TBD 


Approved  by 

Colonel  Michael  L.  McGinnis,  Ph.D. 

Professor  and  Head,  Department  of  Systems  Engineering 


June  2002 


The  Operations  Research  Center  of  Excellence  is  supported  by  the  Assistant  Secretary  of  the 
Army  (Financial  Management  &  Comptroller) 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


I 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  074-0188 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  the  collection  of  information,  including 
suggestions  for  reducing  this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway, 

Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  an  penalty  for  failing  to  comply  with  a  collection  of 
information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 

1.  REPORT  DATE  (DD-MM-YYYY) 
14-06-2002 

2.  REPORT  TYPE 

Technical  Report 

3.  DATES  COVERED  (From  -  To) 

October  2001  -  May  2002 

Hybrid  Value-Utility  Decision  Analysis 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


6.  AUTHOR(S) 

Klimack,  William  K.,  Colonel,  USA 


5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


7.  PERFORMING  ORGANIZATION  NAMES(S)  AND  ADDRESS(S) 

Operations  Research  center  of  Excellence 
Department  of  Systems  Engineering 
United  States  Military  Academy 
West  Point.  NY  10996 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Department  of  Systems  Engineering,  US  Military  Academy 
Attn:  COL  McGinnis 

West  Point,  NY  10996  DSN:  688-2701 

e-mail:  fm0768@usma.edu 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


5f.  WORK  UNIT  NUMBER 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

DSE-TR-02-02 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 


11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


14.  ABSTRACT 

The  Army  faces  complex  decisions  containing  many  variables  and  sources  of  uncertainty.  Under  standard  DA  approaches,  value 
functions  are  employed  under  conditions  of  certainty  and  utility  functions  under  conditions  of  uncertainty.  Utility  functions  are 
unique  to  an  individual,  and  so  must  be  elicited  from  the  decision  maker.  The  time  of  military  decision  makers  is  extremely 
valuable,  and  makes  employment  of  the  utility  approach  problematic. 

The  concept  of  a  hybrid  value-utility  DA  model  is  promulgated.  Such  a  model  may  be  constructed  beginning  with  a  value  model 
based  on  elicitations  from  subject  matter  experts,  e.g.,  staff  officers.  Selected  single  dimensional  value  functions  are  replaced  with 
the  corresponding  utility  functions  to  form  a  hybrid  model.  Substitutions  are  prioritized  through  response  surface  methodology 
applied  to  the  value  functions  of  the  decision  model.  Substitutions  continue  until  the  performance  of  the  hybrid  suitably  estimates 
the  performance  of  the  true  utility  model.  Construction  of  such  a  hybrid  model  minimizes  the  burden  placed  on  the  decision 
maker  while  maintaining  the  benefits  of  DA.  A  sample  case  illustrates  the  elicitation  efficiency  where  an  automatic  target 
‘  recognition  classification  system  choice  attribute  set  was  reduced  from  23  to  eight. 


15.  SUBJECT  TERMS 

Decision  Analysis,  Utility  Theory,  Operations  Research,  Decision  Making,  Response  Surface  Methodology,  Decision  Theory, 
Systems  Engineering. 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT  b.  ABSTRACT 


17.  LIMITATION  OF 

18.  NUMBER 

ABSTRACT 

OF 

PAGES 

UU 

75 

19a.  NAME  OF  RESPONSIBLE  PERSON 

William  K.  Klimack,  COL,  Director,  ORCEN 

19b.  TELEPHONE  NUMBER  (Include  area  code) 
(845)-938-5529  william.klimack@usma.edu 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39-18 


Abstract 


The  Army  faces  complex  decisions  containing  many  variables  and  sources  of  uncertainty. 
Decision  Analysis  (DA)  provides  a  discipline  that  permits  tools  of  quantitative  reasoning  to  be 
employed  to  directly  assist  the  decision  maker.  Under  standard  DA  approaches,  value  functions 
are  employed  under  conditions  of  certainty  and  utility  functions  under  conditions  of  uncertainty. 
Utility  functions  are  unique  to  an  individual,  and  so  must  be  elicited  from  the  decision  maker. 
The  time  of  military  decision  makers  is  extremely  valuable,  and  makes  employment  of  the  utility 
approach  problematic. 

The  concept  of  a  hybrid  value-utility  DA  model  is  promulgated.  Such  a  model  may  be 
constructed  beginning  with  a  value  model  based  on  elicitations  from  subject  matter  experts,  e.g., 
staff  officers.  Selected  single  dimensional  value  functions  are  replaced  with  the  corresponding 
utility  functions  to  form  a  hybrid  model.  Substitutions  are  prioritized  through  response  surface 
methodology  applied  to  the  value  functions  of  the  decision  model.  Substitutions  continue  until 
the  performance  of  the  hybrid  suitably  estimates  the  performance  of  the  true  utility  model. 
Construction  of  such  a  hybrid  model  minimizes  the  burden  placed  on  the  decision  maker  while 
maintaining  the  benefits  of  DA. 

A  sample  case  illustrates  the  elicitation  efficiency  where  an  automatic  target  recognition 
classification  system  choice  attribute  set  was  reduced  from  23  to  eight.  This  reduction  greatly 
improves  the  likelihood  of  senior  leadership  employment  of  decision  analysis  while  providing  a 
requisite  decision  model,  improving  military  decision-making. 
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1  Introduction 


1.1  Purpose 

The  purpose  of  this  technical  report  is  to  provide  the  algorithm  for  adapting  a  value-based 
decision  analysis  model  into  a  model  that  adequately  represents  the  utility  model  in  a  way  that  is 
most  efficient  to  the  decision  maker.  This  hybrid  value-utility  model  has  demonstrated 
efficiencies,  and  is  applicable  for  military  decision-making. 


1.2  Operations  Research  and  Decision  Making 

All  Operations  Research  (OR)  is  concerned  with  making  decisions.  The  introductory 
remarks  in  Introduction  to  Operations  Research  (Hillier  and  Lieberman,  2001:  3)  opines  “to  be 
successful,  OR  must . . .  provide  positive,  understandable  conclusions  to  the  decision  maker(s).” 
Decision  analysis  (DA)  provides  a  framework  for  examining  complex  decisions  made  under 
conditions  of  uncertainty,  particularly  for  single,  non-repeatable  situations.  The  DA  literature 
continues  to  expand,  testifying  to  its  success.  Besides  extensions  to  theory,  DA  has  found 
application  in  many  fields. 

Decision  analysis  provides  a  bridge  between  the  objectivist  and  subjectivist  formulations 
of  probability,  linking  the  decision  maker’s  intrinsic  attitudes  to  the  external  environment  with 
mathematical  tools.  In  a  sense,  DA  has  returned  to  the  roots  of  probability  theory,  to  the  work  of 
Pascal  and  Bernoulli.  In  providing  this  confluence,  DA  provides  a  solid  analytical  foundation  for 
problems  of  increasing  complexity. 

Decision  makers  likely  will  face  increasingly  complex  situations  because  of  increasingly 
complex  technologies,  more  rapid  communications,  faster  management  practices,  and  increasing 
demand  for  organizational  efficiency.  Frequently,  decisions  involve  objectives  that  are  in 
conflict.  Acceptable  alternatives  are  challenging  to  develop.  Decisions  often  involve  intangible 
aspects  and  are  of  interest  to  multiple  groups  with  differing  views  and  may  involve  multiple 
decision  makers.  Often  decisions  involve  expertise  from  several  unrelated  disciplines. 
Uncertainty  also  has  a  significant  impact  on  decisions.  Important  decisions  typically  involve  high 
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stakes,  are  complex  in  nature,  are  multi-disciplinary,  and  need  to  be  defensible  (Keeney,  1982: 
803  -  806).  All  these  aspects  of  decisions  challenge  the  decision  maker,  and  the  operations 
researcher. 


1.3  Multiple  Objective  Decision  Analysis 

Keeney  (1982:  806)  defines  decision  analysis  as  “a  formalization  of  common  sense  for 
situations  which  are  too  complex  for  informal  use  of  common  sense.”  He  also  offers  a  more 
technical  definition:  “a  philosophy,  articulated  by  a  set  of  logical  axioms,  and  a  methodology  and 
collection  of  systematic  procedures,  based  on  those  axioms,  for  responsibly  analyzing  the 
complexities  inherent  in  decision  problems.”  The  term  decision  analysis  is  sometimes  used  more 
broadly  to  refer  to  the  analysis  of  any  decision  (see,  for  example,  [Wallace,  2000]).  In  this 
document,  the  term  decision  analysis  (DA)  refers  to  the  OR  methodologies  derived  from  this 
axiomatic  basis,  those  that  produce  a  preference  function. 

Multiple  Objective  Decision  Analysis  (MOD A)  is  employed  to  permit  the  decision  maker 
to  clarify  in  her  own  mind  which  alternative  she  should  select.  The  intent  of  Multiple  Objective 
Decision  Analysis  is  to  be  prescriptive,  to  provide  what  is  the  most  prudent  course  available  in  a 
bewildering  decision  context,  as  established  from  examination  of  underlying  preference 
components.  Multiple  objective  decision  analysis  is  not  descriptive  in  that  it  does  not  attempt  to 
replicate  how  an  individual  makes  a  decision.  It  is  also  not  normative  in  that  it  accounts  for 
deviation  from  preferences  that  would  be  expected  of  one  who  was  strictly  a  formalist  in 
application  of  mathematical  principles.  (Keeney  and  Raiffa,  1993:  xv).  (A  less  fastidious 
taxonomy  of  decision  making  considers  the  categories  normative  and  prescriptive  to  be  identical. 
For  example,  see  Skinner  [1999:  13].) 

Operations  research  is  credited  as  being  the  first  organized  scientific  discipline  devoted  to 
decision-making.  Emerging  as  a  profession  during  the  Second  World  War,  operations  research 
analysts  played  important  roles  in  developing  effective  tactics  for  air  and  sea  forces.  Postbellum, 
those  skilled  in  OR  techniques  found  applications  in  the  private  sector  and  operations  research 
continued  to  develop.  The  problems  that  were  well  suited  for  application  of  OR  tools  were 
repetitive  in  nature.  Whether  searching  for  enemy  aircraft  or  scheduling  production,  OR 
techniques  were  found  to  be  useful.  Operations  research  flourished  at  the  lower  and  middle 
levels  of  management.  But  executive  decisions  tend  to  be  unique,  not  repetitive,  involve  large 
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portions  of  an  organization’s  resources,  characterized  by  high  degrees  of  uncertainty,  and 
typically  involve  the  decision  maker’s  personal  risk  attitudes.  Operations  research  techniques 
were  found  lacking  for  application  in  the  boardroom.  (Matheson  and  Howard,  1968:  21  -  22) 
Decision  analysis  provides  a  number  of  advantages.  Besides  extending  analysis  beyond 
repetitive  decision  situations,  it  codifies  the  process  of  making  the  actual  decision.  Prior  to  the 
development  of  DA,  many  man-months  of  effort  were  devoted  to  analysis  of  particular  problems. 
This  analysis  may  have  been  development  and  employment  of  a  simulation,  the  application  of 
statistical  tools  to  historical  or  experimental  data,  or  similar  efforts.  The  analytic  results  were 
reduced  to  graphs,  tables,  or  other  conveyances  and  presented  to  the  decision  maker.  The 
decision  maker  then  was  forced  to  absorb  the  information,  process  it  in  some  internal,  usually 
unspecified  manner,  and  then  announce  his  decision.  Decision  analysis  permits  the  tools  of 
operations  research  to  be  brought  to  bear  on  the  decision  itself  to  assist  the  decision  maker. 

Also,  the  decision  analytic  process  itself  has  inherent  advantages.  The  DA  process  forces  the 
decision  maker  to  think  hard  about  her  preferences.  While  this  requires  an  investment  of  time, 
generally  participants  indicate  that  the  exercise  was  beneficial  in  making  them  devote  the  effort 
to  gain  insight  into  the  problem.  (For  example,  see  Felli,  Kochevar,  and  Gritton,  2000:  60  -  61). 
Comer  and  Kirkwood  (1991:  206)  observe,  “Decision  analysis  provides  tools  for  quantitatively 
analyzing  decisions  with  uncertainty  and/or  multiple  conflicting  objectives.  These  tools  are 
especially  useful  when  there  is  limited  directly  relevant  data  so  that  expert  judgment  plays  a 
significant  role.”  Kirkwood  (1992:  37)  notes  that  DA’s  ability  to  perform  tradeoffs  among 
multiple  objectives  makes  it  valuable  for  government  decision  making.  Within  decision  analysis, 
multiattribute  utility  theory  is  considered  the  leading  methodology.  Keeney  and  Raiffa  (1993: 
xi)  state,  “Decision  analysis  is  widely  recognized  as  a  sound  prescriptive  theory.  When  a 
decision  involves  multiple  objectives  -  and  this  is  almost  always  the  case  with  important 
problems  -  multiattribute  utility  theory  forms  the  basic  foundation  for  applying  decision 
analysis.” 

Decision  analysis  is  widely  applicable  to  military  decision-making.  Buede  and  Bresnick 
(1992)  show  that  decision  analysis  is  useful  in  the  materiel  acquisition  process.  Vendor 
engineers  viewed  value  models  distributed  with  the  request  for  proposal  for  the  US  Marine  Corps 
(USMC)  Mobile  Protected  Weapon  System  to  industry  as  helpful.  The  USMC  found  that  all  ten 
conceptual  responses  to  the  request  for  proposal  were  “outstanding”  (115).  Later  work  for  the 
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USMC  involved  adapting  this  value  model  for  off-the-shelf  procurement  of  the  Light  Armored 
Vehicle.  A  utility  model  was  employed  to  determine  the  best  air  defense  system  mix  for  a  US 
Army  heavy  division.  The  authors  used  a  utility  model  to  assess  whether  control  of  the  Patriot 
Air  Defense  System  should  be  transferred  from  the  US  Army  to  the  US  Air  Force.  A  separate 
study  group  conducted  the  effort  in  parallel  with  a  simulation  approach.  The  two  efforts  were  in 
concordance,  the  DA  results  were  more  easily  interpreted  and  more  defensible,  and  the 
simulation  was  an  order  of  magnitude  more  expensive  and  slower  (121).  The  authors  include  a 
list  of  26  other  materiel  acquisition  projects  that  have  successfully  employed  decision  analysis. 
Other  notable  military  decision  analytic  efforts  are  SPACECAST  2020  (Burk  and  Parnell,  1997) 
and  AIR  FORCE  2025  (Jackson,  1996). 

Clemen  (2001:  6-8)  describes  the  decision  analysis  methodology  as  a  five-step  process. 
The  initial  step  is  to  identify  the  decision  situation  and  articulate  objectives.  The  second  step  is  to 
identify  alternatives.  Next  the  problem  is  decomposed  and  modeled.  Three  aspects  are  modeled: 
the  problem  structure,  the  uncertainty,  and  the  preferences.  The  structure  of  the  problem  is 
typically  modeled  using  decision  trees  and  influence  diagrams.  Concepts  of  probability  theory 
are  employed  to  capture  aleatory  aspects  of  the  decision  problem.  Preferences  are  represented 
with  value  or  utility  functions.  This  step,  according  to  Keeney  (1982:  813),  is  unique  to  the 
discipline  of  decision  analysis.  Next,  the  modeling  effort  is  used  to  analyze  the  decision  and  the 
best  alternative  is  chosen.  A  sensitivity  analysis  is  typically  performed  as  the  last  step  to 
determine  the  robustness  of  a  solution.  This  process  is  iterative,  and  repeated  until  an  adequate 
solution  is  determined.  Figure  1  shows  the  process  graphically.  (Alternate,  but  similar, 
constructs  of  the  decision  analysis  methodology  are  available  in  Skinner  [1999:  15]  and  Keeney 
[1982:  807  -  820].) 

1.4  Benefits  of  the  Hybrid  Value-Utility  Approach 

In  complex  problems  the  number  of  attributes  to  be  examined  is  large.  For  example,  the 
United  States  Air  Force  2025  Study,  which  was  commissioned  to  develop  a  recommendation  for 
allocation  of  that  service’s  funds  and  organizational  efforts  towards  development  of  key  system 
concepts  and  their  enabling  technologies,  had  134  attributes  (Jackson,  1996;  and  Parnell,  Conley, 
Jackson,  Lehmkuhl,  and  Andrew,  1998).  (An  effort  for  the  National  Reconnaissance  Office  had 
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Figure  1.  The  Decision  Analysis  Process  (after  Clemen,  2001:  6). 

69  evaluation  measures  [Parnell,  2001].)  The  attributes  ranged  from  operational  performance 
issues  to  technical  feasibility  of  untested  technology.  Clearly  such  large  sets  of  attributes,  which 
draw  on  expertise  from  multiple  disciplines,  is  beyond  the  ability  of  a  single  individual  to  make 
informed  preference  statements.  Instead,  the  judgment  of  subject  matter  experts  must  be 
coalesced  and  presented  to  the  decision  maker.  Under  conditions  of  uncertainty,  common  for 
challenging  decision  situations,  utility  functions  must  be  elicited  personally  from  the  decision 
maker.  Such  an  approach  is  problematic,  as  for  large,  important  decisions;  the  decision  maker 
faces  many  competing  demands  upon  her  time.  A  methodology  that  permits  the  value  model  to 
be  adapted  to  represent  the  decision  maker’s  utility  model  with  the  least  decision  maker 
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interaction  would  be  extremely  valuable  and  critical  for  consideration  of  decision  analysis  in 
such  cases. 


1.5  Sequence  of  Presentation 

The  balance  of  the  report  is  divided  into  five  chapters.  Chapter  2  will  review  the 
literature  regarding  value  and  utility  functions  in  decision  analysis  and  response  surface 
methodology.  Chapter  3  provides  the  theory  supporting  the  hybrid  value-utility  algorithm. 
Chapter  4  presents  the  hybrid  value-utility  algorithm  and  provides  an  example.  Chapter  5 
provides  an  example  application  for  Automatic  Target  Recognition  Classification  System 
selection  and  Chapter  6  briefly  summarizes  the  report. 
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2  Literature  Review 


2.1  General 

As  discussed  in  Chapter  1,  decision  analysis  is  of  recent  development  in  operations  research. 
However,  the  concepts  involved  date  to  the  Enlightenment,  when  mathematical  tools  were 
brought  to  bear  on  decision-making.  That  work  provided  a  foundation  upon  which  modern 
economics  rests,  and  within  the  last  few  decades  has  proven  to  be  the  inchoate  form  of  decision 
analysis. 


2.2  History  of  Utility  and  Value  Theory 

2.2.1  Pascal. 

Hacking  links  the  birth  of  scientific  analysis  of  decision  making  to  Pascal,  and  his  well- 
known  application  of  probability  in  what  is  now  referred  to  as  Pascal’s  Wager  (Hacking,  1975: 

1 1).  Previous  work  with  probability  was  limited  to  solving  puzzles  about  games  of  chance.  Here 
Pascal  wrestles  with  the  concept  that  God  either  is,  or  is  not.  If  God  exists,  one  should  lead  a 
pious  life  leading  to  salvation.  If  not,  one  may  enjoy  one’s  vices  without  regard  to  consequences 
after  death.  However  the  matter  is  unresolved  until  after  death,  and  the  choice  of  how  to  live 
must  be  made  now.  With  the  objective  probability  of  God’s  existence  unknown,  Pascal  focuses 
on  the  idea  of  the  differing  value  of  the  potential  outcomes,  extending  the  ability  to  analyze 
uncertain  situations  beyond  those  posed  about  gaming  tables.  He  is  responsible  for  recognizing 

that  the  expected  value  of  an  uncertain  (discrete)  situation  with  n  outcomes  is  equal  to  , 

where  x,  represents  the  potential  outcomes  and  pt  their  corresponding  probabilities.  Pascal 

vacillated  on  how  to  live  his  life,  sojourning  in  a  monastery,  and  then  abandoning  his  refuge  to 
the  exhilaration  of  the  casinos.  His  final  choice,  as  salvation  was  preferable  to  damnation 
regardless  of  the  probabilities  involved,  was  the  decision  to  return  to  the  monastery  for  good. 
(Bernstein,  1996:  69-71). 
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2.2.2  Bernoulli. 

The  development  of  the  concept  of  utility  theory  is  credited  to  Daniel  Bernoulli.  In  a 
1738  paper  published  by  the  Imperial  Academy  of  Sciences  in  St.  Petersburg,  Bernoulli  states, 
“the  value  of  an  item  must  not  be  based  on  its  price,  but  rather  on  the  utility  that  it  yields.” 
(Bernoulli,  1738/1954:  24).  He  notes  that  under  conditions  of  uncertainty,  the  expected  value  of 
an  outcome  is  determined  by  multiplying  the  probabilities  and  the  respective  “gains.”  But  he 
finds  this  inadequate  for  describing  how  individuals  make  decisions.  He  proposed  that  an 
individual  would  make  a  decision  in  order  to  maximize  expected  utility,  rather  than  maximizing 
the  expected  amount  of  money  (ducats,  for  Bernoulli’s  paper)  or  some  other  objective  measure 
of  worth.  The  calculation  is  made  by  determining  the  mathematical  expectation  employing 

utility  in  place  of  the  units  of  currency,  employed  by  previous  researchers,  yielding  ^ "«(*,  )/>,  > 

where  u  is  the  utility  function.  He  also  asserted  that  for  a  given  decision  situation,  different 
individuals  would  assign  differing  utilities  to  various  possible  outcomes. 

Bernoulli  extended  his  analysis  by  suggesting  that  an  increase  in  utility  resulting  from  a 
gain  in  wealth  would  be  inversely  proportional  to  the  amount  of  previous  possessions.  This  was 
another  key  insight.  He  provided  a  graph  similar  to  Figure  2.  This  inverse  relationship  causes  the 
concavity  shown  by  the  utility  function.  He  also  demonstrated  that  two  individuals  with 
identical  utility  functions,  who  engage  in  a  wager  equally  likely  to  cause  either  to  pay  the  other  a 
fixed  amount,  were  both  facing  a  loss  of  expected  utility.  The  concavity  means  that  the  increase 
in  utility  of  winning  is  less  than  the  reduction  in  utility  of  losing  the  same  amount. 

Bernoulli  used  the  concept  of  utility  to  solve  a  question  originally  posed  by  his  cousin, 
Nicolas  Bernoulli  in  1713.  (With  Daniel  Bernoulli’s  solution,  this  problem  has  become  known 
as  the  St.  Petersburg  Paradox.)  Nicolas  had  developed  a  thought  experiment  where  two 
individuals  agree  to  a  game  between  them  with  no  house  take.  The  first  player  will  flip  a  coin 
repeatedly  until  the  result  is  tails.  Then  he  will  pay  the  second  player  2"  ducats  (or  dollars) 
where  n  is  the  number  of  coin  flips.  Before  the  game  begins,  a  third  player  wishes  to  buy  the 
right  to  play  from  the  second.  Nicolas  notes  that  the  expected  value  of  the  game  in  a  monetary 
sense  is  infinite,  as  the  payoff  increases  infinitely  and  the  probability  of  achieving  another  heads 
never  reaches  zero.  But  most  individuals  would  sell  the  game  for  some  amount  far  less  than 
infinity,  say  between  5  and  20  ducats  (or  dollars).  Daniel  Bernoulli’s  utility  provides  the  answer. 
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The  concavity  of  the  utility  function  keeps  the  perceived  worth  of  the  gamble  far  from  infinity. 
(Bernstein,  1996:  99  -  115). 

2.2.3  Von  Neumann-Morgenstem. 

John  von  Neumann,  in  collaboration  with  Oskar  Morgenstern,  published  the  book  Theory 
of  Games  and  Economic  Behavior.  (Von  Neumann  and  Morgenstern,  1953)  They  proposed  an 
axiomatic  basis  for  utility  theory.  These  axioms  provide  the  necessary  and  sufficient  conditions 
for  the  existence  of  a  utility  function  and  are  presented  in  Appendix  A. 

As  summarized  by  McCord  and  de  Neufville  (1983b:  279),  the  axioms  imply  the 
existence  of  a  cardinal  utility  function  (sometimes  referred  to  as  a  von  Neumann-Morgenstem 
[vNM]  function)  that  orders  alternatives  independent  of  probability  distributions.  Further,  this 
function  may  be  used  to  rank  uncertain  alternatives  by  taking  the  mathematical  expectation  of  the 
vNM  utility.  That  is,  select  the  alternative  with  the  highest  expected  utility  as  determined  by, 

tf  =  Xwi£[“i  (*«■)]  (1) 

1=1 

for  each  alternative,  where  i,  (i  =  1,2, •■•,/),  is  the  index  of  evaluation  measures,  wt  are  the 
relative  weights,  x(  is  the  level  of  the  ith  evaluation  measure,  and  ui  (x;. )  are  the  corresponding 
single  dimensional  utility  levels.  Hence  this  theory  is  referred  to  as  expected  utility  (EU)  theory. 
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Despite  some  criticism  of  EU,  Kimbrough  (1994:  618)  remarked,  “For  prescribing 
behavior,  expected  utility  is  still  by  far  the  most  accepted  theory.”  This  position  is  shared  by 
Edwards  (1992:  xi),  who  points  out  experts  in  the  field  at  a  1992  conference  “unanimously 
endorsed  traditional  SEU  [subjective  expected  utility]  as  the  appropriate  normative  model,  and 
unanimously  agreed  that  people  don’t  act  as  the  model  requires.”  Clearly,  expected  utility  is  the 
preeminent  model  for  non-behavioral  aspects  of  decision  making. 

2.2.4  Measurable  value  functions 

Farquhar  and  Keller  (1989:  205)  attribute  the  “strength  of  preference”  concept  to  Pareto 
and  Frisch.  Stigler  (1965)  provides  a  history.  Also  known  as  “preference  intensity,”  this  is  a 
quaternary  relationship  comparing  preference  differences.  Measurable  value  functions  were 
developed  from  an  axiomatic  basis  as  a  means  of  comparing  an  individual’s  preferences  among 
choices.  (See  Appendix  B  for  the  axioms.)  In  brief,  given  a  set  of  outcomes  {xt ,  x2 , ■  •  • ,  xn } 

arranged  in  order  of  increasing  preference,  comparisons  can  be  made  between  pairs  of  elements. 
For  example,  someone  may  be  indifferent  between  the  respective  increase  in  preference  between 

certain  element  pairs,  (*,.,*,  )  ~  (xk,x,).  This  is  read  as  the  increase  in  preference  from  xi  to  x} 
is  equal  to  the  increase  in  preference  from  xk  to  xt .  By  establishing  such  relationships  and 

normalizing  the  preferences  over  the  unit  interval  (or  any  arbitrary  interval),  a  measurable  value 
function  on  an  interval  scale  may  be  determined.  Note  that  preference  statements  are  made 
under  conditions  of  certainty,  and  therefore  do  not  involve  the  subject’s  risk  attitudes.  Fishbum 
(1967)  provides  a  summary  and  classification  of  measurable  value  function  methods.  For  a 
discussion  of  measurement  theory  (scale  types),  see  Stevens  (1946),  Roberts  (1979)  or  Krantz 
(1971).  A  brief  summary  is  available  in  Forman  and  Gass  (2001:  470). 

2.3  Relationship  of  Value  and  Utility. 

Opinion  regarding  the  relationship  between  value  and  utility  functions  is  varied.  Von 
Winterfeldt  and  Edwards,  (1986:  213)  state  that  there  is  no  significant  distinction.  They  argue 
that  there  are  no  truly  riskless  situations,  so  decision-making  under  certainty  is  never  actually 
encountered.  Secondly,  they  argue  that  concavity  associated  with  risk  aversion  may  be 
explained  by  marginally  decreasing  return  of  value  or  through  a  regret  expression.  They  also 
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argue  that  repetition  reduces  risk  aversion  and  most  choices  are  truly  repetitive  over  a  lifetime. 
Finally  they  opine  that  errors  associated  with  elicitation  of  preference  data  exceed  the 
distinctions  between  value  and  utility  functions.  McCord  and  de  Neufville  (1983a)  provide  an 
empirical  study  supporting  this  position. 

Other  authors  disagree.  Keeney  states  that  value  functions  are  used  under  conditions  of 
certainty,  and  utility  functions  under  conditions  of  uncertainty.  He  also  opines  that  when 
measurable  value  functions  and  utility  functions  for  a  subject  differ,  this  may  indicate  that  the 
subject  has  a  hidden  objective  (1992:  187).  Bouyssou  and  Vansnick  (1988:  110)  observe,  “Many 
authors  have  argued  that  they  see  no  reason  why  lottery  comparisons  should  coincide  with 
preference  difference  comparisons.”  Clemen  (1996:  553)  argues  that  for  decision  analysis  the 
distinctions  are  not  significant  for  applications.  Pennings  and  Smidts  (2000)  state  that  significant 
differences  between  utility  functions  and  value  functions  were  found  by  Krzysztofowicz  (1983a 
and  1983b),  Keller  (1985b),  Smidts  (1997),  and  Weber  and  Milliman  (1997).  Their  “results 
confirm  the  proposition  that  u(x )  and  v(x)  are  different  constructs.”  (1343) 

If  value  and  utility  functions  are  distinct,  it  is  intuitively  appealing  that  they  are  related. 
Both  measure  preference  in  a  given  decision  context.  Utility  has  the  additional  consideration  of 
probabilities  of  various  desired  and  undesired  outcomes,  as  well  as  possibly  other  factors. 
Bouyssou  and  Vansnick  (1988:  109)  state,  “That  some  idea  of  strength  of  preference  interferes 
with  risky  choices  is  hardly  disputable.”  They  continue  (110),  “It  seems  many  features  do 
influence  risky  choice  apart  from  strength  of  preference.  Among  them,  we  feel  regret, . . . 
disappointment,  ...  the  existence  of  a  specific  utility  (or  disutility)  of  gambling,  the 
misperception  of  probabilities  . . .  and  the  avoidance  of  ruin  are  the  most  important  ones.”  They 
state  (109)  “according  to  Bell  [under  certain  conditions]  that  value  is  related  to  utility  either  by 
an  affine  or  an  exponential  transformation.”  They  also  offer  the  observation  that  just  because 
value  and  utility  are  both  on  an  interval  scale  that  does  not  mean  that  there  exists  a  function  that 
is  a  linear  transformation  from  one  to  the  other.  (107) 

Von  Winterfeldt  and  Edwards  (1986:  213)  observe  that  as  of  that  writing,  little  of  the 
literature  addressed  the  relationship  between  value  functions  and  utility  functions.  They 
summarized  the  concept  of  mapping  from  the  decision  space  into  a  preference  space  in  a  graph, 
adapted  in  Figure  3.  An  attribute  is  measured  on  some  scale,  transformed  into  value,  and  then 
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the  corresponding  value  is  transformed  into  utility.  Bypassing  intermediate  steps  is  also 
possible,  as  shown  in  Figure  4. 

Objects  — ►  Natural  or  Constructed  Scale  —►  Value  Scale  ►  Utility  Scale 

Locations  of _  Driving  Distance _  Ratings  of  Relative _ ^  Utilities  of  Values  of 

Apartments  *  (in  miles)  from  Value  of  Driving  Driving  Distances 

Office  Distances 

4  - ►  d(Li) - ►  v(d)  - ►  u (v) 

100  100 


Figure  3.  Mapping  From  Decision  Space  Into  Preference  Space  (Von  Winterfeldt  and  Edwards, 

1986). 

Full  Decomposition 

Objects  ►  Natural  or  Constructed  Scale  Value  Scale-Futility  Scale 

No  Value  Scale  (Keeney  and  Raiffa.  1976) 

Objects  ->•  Natural  or  Constructed  Scale  - ►Utility  Scale 

No  Natural  or  Constructed  Scales  (Bell  and  Raiffa.  1979) 

Objects  - ►  Value  Scale  —►Utility  Scale 

No  Natural  or  Constructed  Scale.  Value  Equals  Utility  (Edward.  1977) 

Objects  - FValue  Scale  =  Utility  Scale 

No  Natural.  Constructed  or  Value  Scales  (Raiffa.  1968) 

Objects  - ►  Utility  Scale 

Figure  4.  Decision  Space  To  Utility  Space  Mapping,  after  von  Winterfeldt  And  Edwards  (1986). 
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Krzysztofowicz  (1983b)  builds  on  the  work  of  Pratt  (1964)  and  makes  the  argument  that 
u  =  w(v)  for  some  transformation  w.  He  developed  various  transforms  for  Pratt’s  measure  of 
local  risk  aversion.  Examining  empirical  data  led  him  to  the  conclusion  that  value  and  utility 
functions  are  different  measures,  both  in  theory  and  as  supported  in  use. 

Keller  (1985b)  examined  the  relationship  between  value  and  utility  functions  in  29 

subjects  for  one  of  four  decision  situations.  She  found  that  generally  v(x)  *  u  (x)  and  that  risk 

attitudes  varied  widely  between  subjects  and  between  attributes  for  a  given  subject.  She  also 
observes  that  her  findings  are  in  disagreement  of  Krzysztofowicz  (1983b),  who  found  the 
exponential  family  to  provide  good  results 

2.4  Decision  Analysis  Methodologies 

A  number  of  textbooks  are  dedicated  to  the  subject  of  decision  analysis  and  offer 
methodological  approaches.  A  selection  of  the  more  influential,  in  order  of  increasing  age,  is 
Clemen  and  Reilly  (2001),  Kirkwood  (1997b),  Keeney  and  Raiffa  (1993),  Kleindorfer, 
Kunreather,  and  Schoemaker  (1993),  Watson  and  Buede  (1987),  and  von  Winterfeldt  and 
Edwards  (1986).  Additionally,  the  work  by  Keeney  (1992a),  while  not  a  textbook,  has  been  very 
influential  in  decision  analysis. 

2.4.1  SMARTS. 

A  number  of  methods  employing  the  underlying  tenets  of  value  and  utility  for  decision 
analysis  have  been  developed.  Within  the  decision  analysis  community,  the  SMARTS  (Simple 
Multiattribute  Rating  Technique  using  Swings)  technique  is  well  known.  Originally  developed 
by  Edwards  (1977)  it  has  undergone  minor  modifications  over  time.  It  is  summarized  in  Edwards 
and  Barron  (1994)  along  with  some  simplifying  modifications  not  discussed  here.  The  steps  of 
SMARTS,  in  the  author’s  terminology,  are: 

1.  Purpose  and  Decision  Makers.  Identify  the  purpose  of  the  decision  and  those  who  will 
make  it. 

2.  Value  Tree  (Hierarchy).  Elicit  a  structured  view  of  the  objectives  for  the  decision. 

3.  Objects  of  Evaluation.  Define  the  alternatives  with  their  attribute  scoring  scales  and 
domain  limits. 
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4.  Objects-by- Attributes  Matrix.  Create  a  matrix  comparing  alternatives  and  scores. 

5.  Dominated  Options.  Remove  from  consideration  all  dominated  alternatives. 

6.  Single-Dimensional  Utilities.  Elicit,  and  then  replace  the  scores  in  the  matrix  with 
utility  levels. 

7.  Rank  Order  Attributes.  Order  in  declining  importance. 

8.  Quantify  Attribute  Weights.  Quantify  in  terms  the  most  important  attribute. 

9.  Decide.  Analyze  the  decision,  employing  an  overall  utility  model.  The  most  common 
is  the  additive  model. 

10.  Determine  the  utility  of  each  alternative  as  given  by 

Uh='LwiE[ui(xhi)]  (2) 

i=i 

where  h  (h  =  1,2,  •••,//)  is  an  index  of  alternatives,  /  (/  =  1,2, — ,/)  is  an  index  of 
attributes,  wt  are  the  weights,  xhi  is  the  ith  evaluation  measure  under  the  hth 
alternative,  and  w(  (xhi )  are  the  corresponding  single  dimensional  utility  levels. 

For  our  purposes,  without  loss  of  generality,  the  preference  functions  will  range  in  the 
unit  interval  and  all  attribute  weights  sum  to  one.  That  is,  u{xt), v(x(.)  e  [0, 1]  =  {r  e  R :  0  <  r  <  1} 

K 

and  3>*=1  with  0  <  wk  <  1  for  all  k .  Further,  in  this  chapter,  increasing  levels  of  x  are 

*=i 

preferred.  Define  x0  as  the  least  preferred  level  of  x  and  x,  as  the  most  preferred.  Then  we  have 
u  (jc0  )  =  v  (x0  )  =  0  and  u  (x» )  =  v  (x, )  =  1 .  These  conventions  are  well-known  in  the  decision 
analysis  literature. 

2.4.2  Utility  Elicitation. 

There  are  several  approaches  for  assessing  the  single  dimensional  utility  functions.  The 
Lock-Step  Procedure  selects  points  in  the  attribute  domain  at  some  fixed  interval  (Kirkwood, 
1997:  233).  At  each  point,  indifference  curves  are  constructed  permitting  a  utility  function  to  be 
established.  A  more  efficient,  and  widely  used,  approach  is  the  Midvalue  Splitting  Technique 
(Kirkwood,  1997b:  233  -  235,  235  -  237).  The  domain  is  bisected,  and  the  utility  elicited  at  that 
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point.  The  two  half-domains  are  similarly  bisected,  and  those  points  elicited.  This  procedure 
may  be  extended  until  the  desired  resolution  is  achieved. 

The  elicitation  of  the  utility  at  these  points  is  frequently  done  employing  one  of  two 
techniques,  the  certainty  equivalent  method  or  the  probability  equivalent  method.  In  both 
methods,  the  decision  maker  is  presented  a  decision  involving  a  choice  between  uncertain 
alternative  A  and  a  certain  alternative  B.  Alternative  A  has  two  possible  outcomes.  One  occurs 
with  probability  p  and  yields  outcome  xt ,  the  most  preferred  level  of  x .  The  other  occurs  with 

probability  (l-  p)  and  produces  x0,  the  least  preferred  level  of  * .  Alternative  B  provides  a 
sure  payoff  of  xB ,  where  x0  <  xB  <  x* .  This  is  shown  in  Figure  5. 

(P) 


Figure  5.  Decision  Diagram.  Squares  represent  decisions,  circles  uncertain  events,  parentheticals 
probabilities,  and  outcomes  expressed  as  the  variable  x. 


In  the  certainty  equivalent  method,  the  probability  p  is  equal  to  0.50 ,  so  Alternative  A  is 
a  50-50  bet  between  the  high  and  low  x  values  under  consideration.  The  decision  maker  is 
presented  various  values  of  xB  until  she  is  ambivalent  in  choosing  between  the  two  alternatives. 
By  the  expected  utility  axioms,  the  utilities  of  both  alternatives  must  be  equal  at  ambivalence. 
Further,  the  utility  of  x*  is  equal  to  one,  and  the  utility  of  x0  is  equal  to  zero,  by  definition.  So 


u(xB)  =  (1- P)u(x0)  +  (p)u(x,) 

(3) 

u(xB )  =  (.5  )u(x0)  +  (,5)u(x*  ) 

(4) 

u(xB)  =  (.  5)(0)  +  (.5)(l)  =  .5 

(5) 
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The  utility  may  next  be  determined  for  the  midpoints  of  the  subregions  [x0,xB]  and  [xB,x*]. 

In  the  probability  equivalent  method,  xB  is  fixed  at  some  value  and  the  probability  p  is  varied 

until  the  decision  maker  is  ambivalent  between  the  alternatives.  This  provides 

u(xB)  =  (l- p)u(x0)  +  (p)u(xt)  (6) 

u(xB ) =  (1  —  p)(0)  +  (p)(l) =  p  (7) 

Since  p  is  known,  the  utility  of  xB  is  known.  The  utilities  of  subregions  are  then  determined  in 

similar  fashion  as  in  the  certainty  equivalent  method. 

One  the  utility  has  been  elicited  at  discrete  points  the  utility  function  is  established.  A 
typical  assumption,  that  of  constant  risk  aversion  permits  the  fitting  of  an  exponential  curve.  In 
practice,  often  only  a  single  midvalue  point  is  used,  and  the  curve  fitted.  (Kirkwood,  1997b: 
235) 


2.4.3  Value  Elicitation. 

Comparing  the  increase  in  preference  for  pairs  of  attribute  levels  permits  value  function 
elicitation.  Often  this  is  done  my  determining  a  midvalue  level.  If  a  decision  maker  is 
indifferent,  for  a  given  price  paid,  to  go  from  x0  to  xm  as  from  xm  to  x„ ,  then  xm  has  value 
equal  to  0.5.  This  from 

-t,. )  ~  (*„.*■  H  [v(*.  )-  v(*0  )]  =  [»(*.)-  v  (*„  )]  (8) 

v(*.)-°=1-v(:0  <9> 

2v(*„)  =  l  (10) 

v(*„ )=)/2  (11) 

Similar  comparisons  over  the  resultant  subregions  permit  additional  value  point  estimates. 


2.4.4  Weight  Elicitation. 

Essentially,  this  is  typically  performed  by  starting  with  all  attributes  at  their  lowest  levels, 
then  “swinging”  one  attribute  at  a  time  from  its  lowest  to  highest  levels.  Take  the  smallest 
reported  increase  in  worth,  and  then  express  the  other  increase  in  terms  of  the  smallest. 
Normalize  the  weights  so  that  they  sum  to  one.  (Kirkwood,  1997b:  68  -  70)  Clemen  (1996:  546 
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-  552)  also  provides  a  description  of  this  technique  as  well  as  pricing  out  and  lottery  weight 
techniques. 

2.5  Sensitivity  Analysis. 

Sensitivity  analysis  has  a  role  in  all  decision  making,  not  just  within  decision  analysis. 

“In  solving  a  problem,  a  sensitivity  analysis  should  be  made  in  order  to  determine  the  variables 
to  which  the  outcome  is  most  sensitive”  (Operations  Analysis  Study  Group,  1977:  12).  A  typical 
decision  analysis  effort  involves  first  constructing  a  deterministic  model  of  the  decision 
opportunity.  Then  the  model  is  examined  to  see  in  where  uncertainty  causes  important  effects. 
Those  variables  where  uncertainty  has  a  significant  impact  are  then  modeled  as  stochastic  while 
the  remaining  variables  are  handled  in  a  deterministic  manner.  The  determination  of  which 
variables  should  become  stochastic  members  of  the  model  is  called  sensitivity  analysis  and  is  and 
is  ascribed  to  Howard  and  sometimes  referred  to  as  the  “Stanford  approach”  (Reilly,  T.,  2000: 
551).  “The  motivation  behind  sensitivity  analysis  is  to  reduce  the  assessment  burden”  (Reilly, 

T.,  2000:  556).  Eschenbach  (1992:  41)  defines  sensitivity  analysis  as  “examining  the  impact  of 
reasonable  changes  in  base-case  assumptions.”  Skinner  (1999:  358)  links  the  employment  of 
sensitivity  analysis  specifically  to  uncertain  aspects  of  a  decision  problem,  rather  than 
assumptions.  Eschenbach  provides  that  the  concern  is  for  the  sensitivity  of  the  response  variable 
to  differing  levels  of  independent  variables.  He  lists  several  considerations  for  sensitivity 
analyses:  (1)  establish  reasonable  bounds  for  independent  variables,  (2)  establish  the  “unit 
impact”  of  these  changes,  (3)  establish  the  maximum  impact  of  the  independent  variables,  and 
(4)  establish  the  change  required  in  each  independent  variable  to  effect  alternative  choices. 

Klimack  (2002)  summarizes  several  approaches  to  sensitivity  analysis  in  DA.  The 
traditional  approach  has  been  termed  the  “Stanford  Approach”  and  involves  perturbing  each 
variable  of  a  deterministic  DA  model,  usually  singly,  and  observing  the  results.  Sensitivity 
analysis  may  be  accomplished  through  a  technique  referred  to  as  Rank  Order  Stability  Analysis 
(ROSA)  and  attributed  to  Einarson,  Arikian,  and  Doyle  (1995)  by  Jolsh  and  Armstrong  (2000: 
537  -  538).  This  approach  is  basically  that  of  a  standard  sensitivity  analysis.  Instead  of  varying 
a  parameter  through  a  range  of  values  of  interest  and  observing  the  response  of  the  objective 
function  and  noting  the  change  in  optimum  alternative,  ROSA  merely  notes  the  range  for  the 
variable  that  retains  the  optimum  alternative.  Dependent  sensitivity  analysis  (DSA),  for 
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sensitivity  analysis  where  the  singular  value  decomposition  of  a  correlation  matrix  is  used  to 
combine  the  independent  variables  into  approximately  uncorrelated  synthetic  variables  Reilly 
(2000).  Bauer,  Parnell  and  Meyers  (1999:  162  -  180),  demonstrated  the  efficacy  of  response 
surface  methodology  (RSM)  for  decision  analysis  sensitivity  analysis.  Response  surface 
methodology  is  “a  collection  of  statistical  and  mathematical  techniques  useful  for  developing, 
improving,  and  optimizing  processes”  (Myers  and  Montgomery,  1995:  1). 


2.5.1  Response  Surface  Methodology. 

Response  surface  methodology  provides  the  ability  to  model  some  system  that  behaves 
according  to  y  =  g  (xvx2,...,xn)+e ,  where  g  is  unknown,  with  y  =  f( xl,x2,...,xn)+e ,  where / 

is  determined  with  empirical  data.  Generally/ is  either  a  first  or  second-degree  polynomial,  and 
forms  the  response  surface.  Figure  6  illustrates  how  a  system  may  be  viewed  as  transforming 
inputs  £  into  some  response  y  or  g  (|, ,  |2 , . . . ,  )  =  y .  Response  surface  methodology  replaces  g 

with  a  low  degree  polynomial/.  Response  surface  methodology  is  frequently  used  as  a  substitute 
for  g  or  to  gain  insight  into  the  important  aspects  of  g.  Response  surface  methodology  is 
sometimes  referred  to  as  regression  analysis  or  meta-modeling.  (Bauer,  2001:  71) 
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Figure  6.  Representation  of  a  System  Modeled  by  RSM. 


Response  surface  methodology  may  be  thought  of  a  polynomial  surface  over  a  region  that 
approximates  the  surface  of  g.  Each  alternative  in  a  decision  would  generally  have  a  differing 
surface  g.  The  independent  variables,  ,  have  upper  and  lower  bounds  Lt  and  Ui  respectively, 
when  considered  across  all  alternatives.  Generally  in  RSM  the  independent  variables  are  coded 


*=2£z4 
'  u 


for  j'  =  l,2,...,n  where  £i0  is  the  midpoint  between  L{  and  Ur  This  maps  Lt 
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and  Ui  to  -1  and  +1,  respectively,  offering  the  advantage  of  working  with  unitless  variables  and 

facilitating  experimental  design.  (Bauer,  Parnell,  Meyers,  1999:  164) 

Response  surface  methodology  produces  a  metamodel  which  models  the  response  of  the 
decision  analysis  model  as  the  input  is  varied.  The  response  is  the  multiattribute  preference 
function,  and  the  input  the  decision  parameters.  The  hypothesized  model  describes  the  response 

y-Po+'tfin+'Zfaxi+i.&’t-  £~h(o,<t!)  (12) 

1=1  i<j  i=\ 

where  the  /?  are  unknown  coefficients.  This  approach  is  analogous  to  a  Taylor’s  series 
expansion  of  degree  2  for  the  true  functional  relationship.  These  coefficients  provide  insight  as 
to  contribution  of  each  variable,  as  well  as  the  strength  of  variable  interactions.  Equation  (12) 
may  be  simplified  by  omitting  the  xf  term  and  the  xtXj  interaction  term  may  also  to  neglected. 

As  simplified,  the  expression  is  easier  to  solve  and  so  is  often  used  as  a  screening  device  early  in 
the  RSM  process.  Employing  the  complete  Equation  (12)  near  the  subregion  of  concern 
provides  a  better  fit  of  the  approximated  surface.  The  least  squares  estimated  coefficients  of  the 
hypothesized  model 


y  =  b, b  +  X  Vi  +Yjbiixix) +  X  V? 

1=1  1<  j  1=1 

(13) 

is,  in  vector  notation, 

b  =  (X'X)-'  X'y 

(14) 

The  fitted  surface  is  provided  by 

y  =  Xb 

(15) 

and  residual  error  is 

e  =  y-y 

(16) 

The  coefficient  provides  a  ready  indicator  of  the  contribution  for  the  associated  variable  and 
statistical  tests  provide  a  measure  of  significance.  (Myers  and  Montgomery,  1995:  16-21) 
Bauer,  Parnell,  and  Meyers  present  Figure  7,  a  graphical  representation  of  iterative 
application  of  RSM  to  a  DA  problem  (1999:  165  -  166).  To  reduce  the  analytic  burden, 
variables  that  are  believed  to  behave  similarly  may  be  grouped  and  a  RSM  screening  conducted. 
Those  groups  that  have  significant  effects  are  then  screened  as  individual  factors.  Significant 
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variables  are  then  modeled  with  a  first  order  equation  and  the  fit  is  tested.  If  the  fit  is 
insufficient,  a  second  order  model  is  employed. 


Group  1  -[ 

Group  2  ■[ 

Group  3  — 


Group  1 


{ 


Group  3  — 


Figure  7.  RSM  Paradigm  (After  Bauer,  Parnell,  and  Meyers,  [1999:  166]). 


2.5.2  Group  Screening. 

As  Bauer,  Parnell,  and  Meyers  (1999)  employ  group  screening,  it  deserves  additional 
discussion  beyond  a  review  of  RSM.  Meyers  and  Montgomery  (1995)  fail  to  mention  group 
screening  in  their  RSM  text.  A  screening  design  in  experimentation  is  a  selection  of 
experimental  inputs  that  permit  establishment  of  which  factors  are  significant  while  minimizing 
the  number  of  experimental  iterations  (usually  referred  to  as  runs  in  modeling)  and  so  reducing 
associated  costs.  Large  numbers  of  factors  force  a  prohibitively  large  number  of  runs  or,  when 
the  number  of  runs  is  constrained,  causes  factors  to  be  confounded  and  effects  masked.  Madu 
and  Kuei  (1992:  96)  caution  that  screening  large  numbers  of  variables  is  to  be  avoided.  Group 
screening  reduces  the  number  of  factors  considered  by  consolidating  them  into  groups.  These 
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groups  are  then  manipulated  simultaneously  so  the  experimental  design  has  been  reduced  from 
the  total  number  of  factors  to  the  number  of  groups. 

In  the  procedure,  stated  simply,  variables  are  first  aggregated  into  groups.  Various 
authors  have  developed  recommended  differing  grouping  assumptions  and  strategies.  Then  an 
acceptable  experimental  design  based  on  the  groups  is  selected.  When  the  runs  are  complete,  all 
variables  contained  in  the  groups  found  not  to  be  significant  are  assumed  to  be  non-significant. 
Significant  groups  are  decomposed  and  new  groups  formed,  if  required,  and  the  process 
repeated.  Their  efforts  are  shown  graphically  in  Figure  7. 

While  standard  decision  analysis  sensitivity  analysis  identifies  which  variables,  without 
interaction,  cause  the  solution  to  be  sensitive,  and  DSA  addresses  interactions  employing 
synthetic  variables,  RSM  provides  a  quantitative  estimate  of  the  impact  of  each  variable  and 
interactions.  Sensitivity  analysis  in  the  decision  analysis  literature  excludes  discussion  of 
preference  functions.  The  literature  does  not  address  the  issue,  although  it  does  provide  evidence 
that  the  issue  is  of  import.  Ghosh  and  Ray  (1997)  found  that  decision  maker’s  choices  are  both  a 
function  of  risk  attitudes  and  ambiguity  tolerance.  Additionally,  their  results  suggest  that  the 
decision  maker  often  interprets  ambiguity  in  a  decision  situation  as  risk.  This  indicates  that  the 
preference  function  likely  involves  some  inherent  uncertainty. 

Besides  the  uncertainty  present  within  the  decision  maker,  elicitation  sessions  are 
vulnerable  to  human  perception  frailties.  An  example  of  this  includes  framing,  where  the  same 
situation  presented  in  differing  manners  elicits  consistently  differing  preferences.  Anchoring  is 
the  phenomenon  in  which  an  individual  presented  with  some  estimate  of  a  parameter  of  interest 
will  often  provide  biased  elicitation,  assessing  higher  probabilities  against  the  estimate  while 
significantly  underestimating  probabilities  of  severe  events.  Further,  it  is  well  documented  that 
individuals  will  demonstrate  more  confidence  in  objective  probabilities  than  subjective 
probabilities  without  discernable  justification.  See  Kirkwood  (1997b:  1 10  -  127;  299  -  320)  for 
discussion  of  these  issues. 
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3  Hybrid  Value-Utility  Decision  Analysis  Models 


3.1  General 

As  discussed  previously,  decision  analysis  models  may  be  utility-function-based,  or 
value-function-based.  Klimack  (2002b)  showed  that  the  two  models  differ  significantly  for 
military  decision-making.  The  utility-based  model  is  appropriate  when  outcomes  are  not  certain, 
as  is  typical  for  difficult  decisions.  As  utility  functions  incorporate  the  risk  attitudes  of  an 
individual,  they  are  specific  to  an  individual.  Therefore  construction  of  utility  models  requires  a 
significant  amount  of  the  decision  maker’s  time.  For  large  decision  problems,  this  personal 
commitment  on  the  part  of  the  decision  maker  is  onerous  and  is  a  barrier  to  the  use  of  multiple 
objective  decision  analysis.  Value-based  models,  appropriate  under  conditions  of  certainty,  may 
be  constructed  through  consultation  with  appropriate  experts  (e.g.,  a  military  staff)  rather  than 
the  decision  maker.  Consultation  with  subject  matter  experts  is  generally  required  in  any 
complex  problem,  regardless  of  methodology  applied. 

If  a  utility-based  model  may  be  approximated  by  modification  of  a  value-based  model, 
reduction  of  the  time  demands  on  the  decision  maker  may  be  realized.  For  large,  important 
decisions,  these  time  demands  are  significant.  Reducing  the  time  required  of  the  decision  maker 
improves  the  acceptance  of  the  procedure  and  improves  the  probability  that  decision  analysis 
will  be  employed.  This  chapter  will  first  review  pertinent  background  information.  Then  an 
algorithm  will  be  presented  for  construction  of  the  hybrid  model. 

3.2  Background 

Decision  analysis  models  consist  of  a  set  of  evaluation  criteria  for  which  preference 
functions  are  used  to  assess  the  decision  maker’s  strength  of  preference  for  various  levels  of  the 
criteria,  and  often  also  attitudes  regarding  the  uncertainty  of  the  decision  situation.  In  a  typical 
value-focused  approach  (Keeney,  1992),  the  evaluation  criteria  are  determined  through 
decomposition  of  the  decision  maker’s  strategic  priorities  until  measurable  criteria,  the 
evaluation  measures,  are  obtained.  This  inverted  dendritic  structure,  the  value  hierarchy,  is 


22 


shown  in  the  upper  portion  of  Figure  8.  Each  evaluation  measure  is  assessed  using  a  single 
dimensional  preference  function.  Under  conditions  of  certainty,  the  preference  function  is 
referred  to  as  a  value  function  and  measures  the  decision  maker’s  strength  of  preference.  Under 
uncertainty,  the  preference  function  is  referred  to  as  a  utility  function  and  measures  both  the 
decision  maker’s  strength  of  preference  and  her  attitude  toward  the  risk  inherent  in  the  uncertain 
decision  context.  The  preference  functions  are  contained  in  the  second  from  bottom  row  of 
Figure  8.  A  multiattribute  (multidimensional)  function  employing  relative  weights  (relative 
importance)  of  the  various  evaluation  measures  combines  the  single  dimensional  preference 
functions  and  completes  the  model,  permitting  evaluation  of  alternatives.  Figure  8  represents  a 
common  view  of  decision  analysis  models,  which  assumes  perfect  information.  (In  fairness,  it 
must  be  observed  that  sensitivity  analysis  is  often  used  to  examine  the  assumption  of  perfect 
information.) 
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Figure  8.  Common  Decision  Analysis  Model  Structure.  The  bottom  tier  represents  the 
evaluation  measures  and  the  next  tier  the  preference  functions.  The  hierarchy  represents  the 

decomposed  values  of  the  decision  maker. 


A  multiattribute  function  for  a  utility  model  is  often  written  in  the  additive  form.  This  is 

Uj  =  'ZE[wiui(xi)]  (17) 

1=1 

where  Uj  is  the  utility  of  the  jlh  alternative,  w,  is  the  weight  (relative  importance)  of  the  ith 
evaluation  measure,  and  w(  (x( )  is  the  single  dimensional  utility  function  for  the  ith  evaluation 
measure,  x.  is  the  random  variable  for  the  ith  evaluation  measure,  and  /  is  the  number  of 
evaluation  measures. 
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Equation  (17)  may  be  rewritten  in  terms  of  the  utility  probability  density  function,  or 


e[uj]=e  X (xi )» «1 )  =sLwiE [f*iui (*< ) » K ). 


(18) 


L  «= 1  J  i=i 

where  f"  is  the  probability  density  function  of  the  ith  utility  function,  and  &'  is  a  vector  of 
parameters  of  the  estimated  evaluation  measure  distributions.  The  DA  problem  may  be  stated  as 
maximizing  £[£/;]  through  the  selection  of  the  y'th  alternative.  More  formally,  we  take  the 

maximum 


such  that 


Max  E[Uj] 

(19) 

:{L Ix,U2,...,Uj} 

(20) 

.  1=1 

(21) 

1  =  1,...,/ 

(22) 

(23) 

ve  multiattribute  function  is 

/ 

=Xw<v<(x<) 

(24) 

1=1 


where  Vy  is  the  value  of  the  y'th  alternative  and  v(-  (xt )  is  the  single  dimensional  value  function  for 

the  ith  evaluation  measure.  Equation  (24)  contains  no  expectation  operator,  as  it  is  intended  for 
us  under  conditions  of  certainty.  However,  using  value  functions  under  uncertainty  produces  a 
value  pdf  that  behaves  similarly  to  the  utility  pdf.  Equation  (24)  then  becomes 

V;=X£[w<v<00]  (25) 

i=l 

In  analogous  fashion,  an  objective  value  function  may  be  constructed 

MaxE[V,]  (26) 

such  that 

VjefyuV2,...,Vj}  (27) 
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(28) 


1=  1 

*  =  1 . /  (29) 

j  =  (30) 

where  f'  is  the  probability  density  function  of  the  zth  value  function,  and  a'.  is  a  vector  of 
parameters  of  the  estimated  evaluation  measure  distributions.  The  DA  problem  may  be  stated  as 
maximizing  E  |  J  through  the  selection  of  the  yth  alternative. 

In  uncertain  decision  contexts  Equation  (18)  is  appropriate  but  for  large  problems  of  great 
import  the  ui  (x, )  are  difficult  to  elicit  from  the  decision  maker  because  of  the  time  involved.  As 

risk  attitudes  vary  within  an  organization,  it  is  not  acceptable  to  use  a  surrogate  for  the  decision 
maker  to  elicit  utility  functions.  Arguably,  an  organization  should  have  a  consensus  for  values  in 
a  decision  situation,  so  parameters  for  Equation  (24)  are  may  be  elicited  from  subject  matter 
experts.  Modifying  Equation  (24)  provides  an  approach  to  estimate  Equation  (17)  to  within 
some  specified  accuracy.  The  advantage  of  this  approach  is  the  reduction  of  the  elicitation 
burden  on  the  decision  maker.  This  onerous  burden  sometimes  either  precludes  employing  a 
utility  approach  or  causes  the  selection  of  non-decision  analysis  techniques,  and  has  been 
identified  as  a  limitation  of  this  methodology. 

The  proposed  procedure  is  depicted  graphically  in  Figure  9.  Starting  with  the  value 
model,  a  hybrid  model  is  created  by  replacing  certain  single  dimension  value  functions  with  the 
corresponding  utility  functions.  The  resultant  hybrid  model  is  then  used  as  would  the  utility 
model  to  analyze  the  decision  situation. 

Typically,  the  initial  step  of  building  the  value  model  is  done  through  interview  of  the 
decision  maker.  As  we  wish  to  minimize  the  time  required  of  this  individual,  we  will  construct 
the  value  model  using  appropriate  subject  matter  experts.  The  decision  maker  then  should 
validate  the  value  model.  As  we  are  concerned  with  complex  decision  situations,  often  we 
would  expect  the  number  of  evaluation  measures  to  be  large.  Group  screening  is  a  method  that 
efficiently  examines  large  numbers  of  variables.  We  anticipate  that  group  screening  will  be 
employed  as  part  of  the  step  determining  significance  of  the  evaluation  measures.  Further 
development  of  the  methodology  outlined  in  Figure  9  is  provided  in  the  next  chapter. 


£[V,]  =  £ 
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Figure  9.  Basic  Hybrid  Value-Utility  Model  Approach. 


4  A  Hybrid  Value-Utility  Algorithm 


4.1  Algorithm  Development. 

A  method  of  approximating  Equation  (17)  is  desired.  Beginning  with  Equation  (24),  one 
evaluation  measure’s  value  function,  vk  (xk),  is  replaced  with  the  appropriate  utility  function, 


uk  (xk ) .  The  model  becomes 

]  =  £  £  wj'iy,  (xi),a'])+  wjk  (uk  (**),&') 


_f=l  ti*k 


(31) 


where  the  ifcth  evaluation  measure  value  function  has  been  replaced  with  the  corresponding  utility 
function  and  is  an  estimate  of  Uk  after  one  substitution. 


Equation  (31)  then  becomes,  after  m  substitutions, 

£[(?<">]  =  E  £[0  -  K, (x, ),«()+  K, w.f’fc  (*, )•««)] 


(32) 


where  Kt  is  an  indicator  variable  corresponding  to  the  substitutions  defined  for  the  ith  evaluation 
measure  as 


K,  = 


[ 0  when  vf  (xi )  remains, 

I  1  when  ui  (x(. )  substituted 


(33) 


This  substitution  protocol  continues  in  an  iterative  fashion  until  the  estimated  utility,  U)  ’  is 
sufficiently  close  to  U .  This  may  be  stated  as  |  E  J  -  E  [ Uk  ]|  <  8 ,  where  8  is  the  level  of 

accuracy  desired.  Then  £  ~  £[[7^] .  Equation  (32)  is  a  hybrid  value-utility  model  that 

estimates  the  true  utility  model  behavior. 


4.2  Substitution  order 

Two  additional  considerations  must  be  addressed  in  order  to  develop  an  algorithm.  These 
are  the  order  of  the  substitutions  of  single  dimensional  utility  for  single  dimensional  value 
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functions,  and  when  the  substitutions  may  be  halted.  In  general,  the  substitutions  should  be 
made  in  order  of  decreasing  impact  upon  the  hybrid  model,  equation  (32).  The  coefficients  of 
the  regression  model  produced  by  RSM  indicate  the  potential  contribution  of  each  model 
variable  and  permit  the  prioritization  of  substitution  of  the  evaluation  measures.  The 
substitutions  may  be  stopped  when  sufficient  accuracy  is  achieved,  as  discussed  below. 

Frequently,  utility  functions  are  approximated  employing  parametric  functions  fitted  to 
the  data  (Kirkwood,  1997:  65).  When  the  exponential  function, 

“,  (v,  (*,))  =  "  g  -i/  (34) 

\-e/p‘ 

is  used  to  approximate  a  utility  function  the  exponential  constant,  p ,  may  be  modified  to 
examine  an  envelope  of  utility  curves.  Setting  p,  =  °°  for  u{  (v  (x, ))  establishes  that  there  are  no 

risk  attitudes  and  therefore  m,  (v  (x  ))  =  v,.  (x( ) .  Varying  p,.  permits  employment  of  response 


surface  methodology  to  determine  whether  each  m,  (jc,  )  potentially  exerts  a  significant  effect  on 

the  hybrid  model.  Kirkwood  (1997b:  66)  indicates  that  the  absolute  value  of  p  is  typically 

,  ,  „  ,  ,  .  ^  max x -minx 

greater  than  one-tenth  of  the  evaluation  measure  domain,  or  p(.  > - yy - -  .  The  region 

bounded  this  relationship  is  illustrated  in  Figure  10.  Accepting  the  limits  proposed  by  Kirkwood 
means  that  if  we  examine  the  two  w,  (x, )  for  p  =  ±0. 1 ,  and  they  do  not  significantly  affect  the 

hybrid  model,  then  the  preference  function  for  the  i  th  evaluation  measure  may  remain  a  value 
function.  We  do  not  expect  the  utility  function  for  that  evaluation  measure  to  significantly  affect 
the  hybrid  model. 


4.3  Stopping  Criteria 

Replacing  all  potentially  significant  evaluation  measures  will  ensure  that  the  hybrid 
model  adequately  represents  the  utility  model.  However,  it  is  desired  to  halt  the  iterative 

substitutions  when  -  £[*7,1  ^  8  >  which  may  occur  before  all  the  evaluation  measures 
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- Infinity 

-B-O.l 
-A-  -0.1 


v,0.) 

Figure  10.  Exponential  Utility  Function  Limits.  The  straight  line,  where  m,  (x,)  =  v,  (x, ),  occurs 
when  p  =  °° .  The  other  curves  show  (x, )  when  p  =  ±0. 1 . 


have  been  converted.  We  may  write  the  problem  as  a  mathematical  programming  problem.  The 

optimization  model  is: 

/ 

min  m  =  ^jKi 

1=1 

(35) 

subject  to 

|E[y!">]-E[t/,.]  <<SV; 

(36) 

Se  [0,1] 

(37) 

However,  E  [t/;  J  is  unknown,  so  it  is  not  directly  possible  to  determine  when 
is  within  S  of  E[c/y  ].  When  there  is  consistent  risk  aversion  (risk  affinity)  then  the  set  of  all 

possible  Sj ,  is  bounded  below  (above)  by  Vj  and  above  (below)  by  E[t/y].  See 

Figure  1 1  for  examples  of  behavior  of  preference  function  under  various  risk  attitudes.  When 
risk  aversion  is  present  for  the  ith  evaluation  measure,  w(.  (v(.  (x, ))  >  v,  (x, ) .  The  reverse  is  found 
under  risk  seeking  behavior.  Mixed  risk  attitudes  preclude  such  general  conclusions 
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Mixed  Risk  Attitude 


Figure  11.  Risk  Attitudinal  Effects  on  the  Relationship  of  Value  and  Utility  Functions. 

A  useful  measure  is  the  substitution  distance,  the  difference  between  successive 
substitutions.  Given  a  series  of  substitutions  to  create  hybrid  value-utility  models  for  the  jth 

alternative,  Sj  =  {vj , OJ j ,  the  distance  between  successive  substitutions  is  given 
by  ,  defined  as 

/l{m)  =  |£[f/y(m)]-£[f/7(m_1)]|  (38) 

where  m  indicates  the  number  of  substitutions,  and  =  £’[V/] . 

When  the  set  Sy  is  ordered  with  decreasing  intervals  between  iterative  estimates, 

/ljm)  <  Ajm“l),Vme  {2,3,...,/},  then  Sj  is  a  partially  ordered  set.  Under  this  condition,  when 
(/  <  8 ,  then  |£'[ujm)]-£'[UJ.]|  <  8 .  When  (/  <  8  is  encountered  the 

substitutions  of  single  dimensional  utility  functions  for  value  functions  is  halted  and  the  current 
£;[c/jra)]  accepted  as  accurately  estimating  E  [t/;.  J . 

There  are  a  number  of  potential  alternatives  of  interest,  not  just  alternative  j.  Define  the 
vectors  U  and  U^ 


'u; 

u  <-> 

U  = 

uj 

and  U(m)  = 

uf 

Uj. 
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Clearly  it  is  not  necessary  to  continue  iterations,  and  so  to  increment  m,  when 
[u^]  —  E  [Uy  ]|  <8\/j .  What  is  of  interest  is,  naturally,  the  alternative  with  the  greatest 

utility.  Therefore  we  are  satisfied  when  the  alternative  with  the  greatest  utility  is  alone  within 
the  accepted  tolerance.  Define  =  max -  ■  ■  ,Ujm) }  where  m  is  the  most  recent 

iteration.  Define  /lfm)  =  where  *  represents  the  evaluation  measure  with 

the  greatest  utility  in  the  last,  mth,  iteration.  The  mathematical  programming  formulation  now 
becomes: 

/ 

min  m  =  ^  Kt  (40) 

i=i 

subject  to 

{l-rn)tim)<S  (41) 

Se[ 0,1]  (42) 

When  Sj  is  partially  ordered,  m  is  minimized  when  equation  (41)  is  first  found  to  hold. 
Response  surface  methodology  provides  the  sufficient  conditions  for  partial  ordering  of  the 
substitution  distances.  Stop  the  algorithm  when  (I  -m)/\im)  <  S  and  accept  as 

accurately  estimating  E[Ut]  and  the  alternative  corresponding  to  u[m)  as  being  the  optimal 
choice. 


4.4  Group  Screening 

As  stated  previously,  the  class  of  decision  analysis  problems  considered  on  which  these 
techniques  are  likely  to  be  beneficial  are  those  with  a  large  number  of  evaluation  measures.  For 
such  problems,  group  screening  is  likely  to  be  required.  The  group  screening  employment  is  as 
traditionally  employed  as  presented  in  Chapter  2.  Once  the  screening  is  completed,  the 
information  must  be  re-aggregated,  which  is  a  new  contribution. 

Employing  groups  screening  permits  examination  of  a  large  number  of  variables  through 
decomposing  the  independent  variables  into  subsets.  Once  the  groups  screening  is  completed,  it 
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is  desired  to  reassemble  the  variables  into  a  single  set,  rank  ordered  by  the  degree  of  affect  each 
has  on  the  model.  The  utility  model  is 

#  =  SWA(V»(*<))  (43) 

1=1 

where,  with  the  assumption  of  the  exponential  single  dimensional  model,  the  /th  evaluation 
measure  is  determined  by 

~V,(% 

-y*  <44) 

l-e/p‘ 

When  the  kth  element  of  (43)  is  examined  separately,  the  model  becomes 

U  =  X  wfi,  (v,.  (xt ))  +  wkuk  (vk  (xk ))  +  X  cixi  (45) 

i-  1  i=k+] 

As  equation  (43)  may  be  represented  by  a  fit  of  a  linear  model  based  on  the  exponential  constant 

f/  =  Co  +  i>,A  (46) 

i=\ 

the  kth  element  of  (45)  may  be  decomposed  from  (46)  and  modeled  by  a  least  squares  model 

k — 1  n 

Uk  =  C0  +  X CiPi  +  CkPk  +  X  ClPl  (47) 

i=l  i=k+\ 

The  coefficient  ck  provides  a  measure  of  importance  of  pk  relative  to  the  other  p. . 

If  the  kth  element  has  been  selected  to  be  treated  as  a  group  of  variables  for  screening, 
then  it  becomes  necessary  to  decompose  the  kth  group  when  it  is  determined  to  be  significant. 
Because  the  kth  group  may  also  be  modeled  by 

n 

Uk  =  Ck0  +  lckiPki  (48) 

1=1 

the  contribution  to  the  overall  model  for  the  pkm  th  variable  is  ckckm .  Similar  combinations  are 

made  for  all  grouped  variable  coefficients  to  permit  comparison  of  evaluation  measure  priorities. 
The  relative  importance  of  utility  functions  for  the  evaluation  measures  is  provided  by  the 
corresponding  •■•ck^fkr  where  r  is  the  number  of  screening  sets  in  which  the  kth 

evaluation  measure  has  been  grouped.  Rank  ordering  the  evaluation  measures  by  relative 
importance  provides  the  order  for  the  substitutions  from  single  dimensional  value  function  to 
single  dimensional  utility  functions  when  the  hybrid  model  is  created. 
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4.5  The  Detailed  Algorithm 

Refining  the  basic  concept  provided  in  Figure  9  by  minimizing  the  interactions  required 
with  the  decision  maker,  incorporating  group  screening,  and  using  response  surface  methodology 
to  assess  potential  significance  provides  the  algorithm  presented  in  Figure  12. 

The  steps  of  this  protocol  are: 

1.  Build  Value  Model. 

2.  Establish  the  multiattribute  functional  form.  Typically  the  simple  additive  form  is 
used,  but  the  algorithm  is  independent  of  the  multiattribute  functional  form.  A 
constraint  on  the  algorithm  is  that  the  value  model  and  the  utility  model  must  be  of 
the  same  form. 

3.  Elicit  value  hierarchy  and  evaluation  measure  weights,  w(. ,  from  subject  matter 

experts.  This  avoids  a  significant  time  burden  being  placed  upon  the  decision  maker. 

4.  Verify  the  value  hierarchy  and  evaluation  measure  weights  with  decision  maker. 
Clearly  the  model  must  be  acceptable  to  the  decision  maker.  Verification  at  this  point 
provides  efficient  elicitation  of  the  single  dimensional  value  functions. 

5.  Elicit  V;  (jc,  )V  i  from  subject  matter  experts. 

6.  Determine  potential  impact  of  single  dimensional  utility  functions. 

a.  Group  v(  (jc,.  ) .  Grouping  is  done  to  permit  group  screening,  which  allows 

examination  of  large  numbers  of  evaluation  measures.  The  value  hierarchy 
provides  a  ready  framework  for  grouping  of  variables.  An  obvious  approach 
is  to  group  variables  at  the  highest  partitioning.  Each  group  may  be 
subsequently  examined,  employing  group  screening  as  required.  An 
additional  consideration  is  that  if  any  interactions  are  believed  to  be  present, 
the  groupings  should  not  separate  those  variables. 

b.  Develop  and  execute  experimental  design.  Design  of  experiments  permits 
the  significance  of  the  predictor  variables  to  be  assessed  with  the  minimum 
number  of  experimental  runs.  A  run  in  this  sense,  is  setting  the  variables  at 
some  level  and  observing  the  expected  utilities  of  the  various  alternatives. 

The  variables  of  interest  with  respect  to  the  hybrid  model  are  the  exponential 
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Build  Value  Model 


Establish  Multiattribute  Functional  Form 


Figure  12.  Steps  to  Create  Hybrid  Value-Utility  Model. 


constants  of  the  w,.  (v,  (jc,.  ))  functions.  As  the  exponential  constants  assume 

values  of  p  =  {-0. 1,  «=, 0.  l},  the  fractional  factorial  with  center  point  designs 

are  attractive.  The  experimental  design  is  complicated  by  the  consideration 
of  the  set  of  alternatives.  The  alternative  choice  may  be  included  as  a 
variable  in  the  experimental  design.  If  the  set  is  large,  the  number  of 
available  experimental  designs  is  small.  This  step  and  the  following  step  are 
repeated  for  each  grouping  of  variables. 

c.  Employ  RSM.  Examine  potential  contribution  of  w,  )  in  the  current 

model  in  place  of  v,.  (jc,. )  by  employing  response  surface  methodology  to 

examine  m,.  (vt  (jc,. )) .  This  analysis  provides  regression  coefficients  showing 

the  contribution  of  each  variable  as  well  as  significance  information.  This 
provides  data  that  facilitates  determining  the  evaluation  measures  that  need  to 
be  considered  for  conversion  into  utility,  at  some  defined  alpha  level,  as  well 
as  straightforward  prioritization  information.  This  step  is  done  for  each 
grouping  of  variables. 

d.  Ungroup  v,.  (jc,.  ) .  The  RSM  information  from  the  previous  step  is 

ungrouped  to  provide  a  common  comparison  for  all  evaluation  measures. 
Linear  regression  coefficients  are  multiplied  by  the  coefficients  for  all  parent 
groups  to  provide  a  single  rank-ordered  list  of  the  potentially  significant 
evaluation  measures. 

7.  Replace  potentially  significant  evaluation  measures  with  utility  functions.  In 
priority  order,  perform  iterative  substitutions.  This  is  done  by: 

8.  Set  iteration  counter  m  =  1 

9.  Select  the  remaining  v(.  (jc,.  )  from  the  list  of  potentially  significant  evaluation 
measures  with  the  greatest  coefficients  from  the  RSM  model. 

10.  Elicit  ut  (jc,  )  for  this  evaluation  measure. 

11.  Determine  V/  by  substituting  the  w,.  (jc;  )  from  the  previous  step. 
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12.  If  m  =  1  or  if  either  stopping  criteria  is  not  met,  increment  m  =  m  +  \  ancj  return  to 
step  b.  Otherwise  stop  and  £^t/»(m) ]  ~E[Ut].  The  stopping  criteria  are: 

a.  All  significant  evaluation  measures,  as  determined  by  RSM,  have  been 
converted  from  v,  (x, )  to  w,  (xi ) . 

b.  is  sufficiently  close  to 

13.  The  hybrid  model  is  now  used  as  would  a  standard  utility  model  to: 

a.  Select  the  best  alternative,  ,  from  an  expected  value  standpoint,  and 

b.  Perform  a  sensitivity  analysis. 
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5  Demonstration  of  Value-Utility  Hybridization 


Klimack  (2002b)  provides  an  example  of  the  employment  of  the  hybrid  value-utility 
approach.  An  illustrative  summary  is  provided  here.  Klimack,  Bassham,  and  Bauer  (2002a) 
applied  decision  analysis  to  an  Automatic  Target  Recognition  (ATR)  problem.  This  application 
of  decision  analysis  included  eliciting  both  single  dimensional  value  and  utility  functions  from 
an  expert  in  the  field.  This  provided  an  opportunity  to  employ  the  Value-Utility  Hybridization 
methodology. 


5.1  Background. 

Automatic  Target  Recognition  (ATR)  is  a  processing  problem  where  an  image  is 
examined  to  identify  or  otherwise  characterize  targets.  All  ATR  classification  systems  (CSs) 
have  a  number  of  desirable  traits  that  may  be  used  as  evaluation  measures  when  selecting  from 
among  candidate  systems.  In  general,  these  measures  have  not  applied  in  total,  but  specific 
measures  have  been  selected  when  considering  alternatives  for  a  specific  program.  A  decision 
analysis  approach  was  employed.  A  value  model  was  constructed.  The  value  hierarchy  is 
presented  in  Figure  13,  and  captures  the  motivators  important  to  the  decision  maker.  Besides  the 
values,  the  problem  is  affected  by  the  two  mission  profiles  (combat  identification  [CID]  and 
intelligence/surveillance/reconnaissance  [ISR])  under  which  the  ATR  CS  would  be  employed. 
Additionally  there  are  three  alternatives  under  consideration.  Employing  single  dimensional 
value  functions  and  single  dimensional  utility  functions  produced  differing  results,  indicating 
that  the  problem  was  sensitive  to  preference  function  construction. 

5.2  Employing  RSM  to  Prioritize  Model  Parameters 

The  ATR  DA  problem  had  23  evaluation  measures,  23  corresponding  weights,  two 
mission  profiles  (combat  identification  [CID]  and  intelligence/surveillance/reconnaissance 
[ISR]),  and  three  alternatives.  This  was  a  moderately  large  number  of  parameters.  Weights  were 
perturbed  using  the  procedure  provided  by  Kirkwood  (1997b:  83  -  84)  as  implemented  by  Bauer, 
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Figure  13.  The  Value  Hierarchy  for  the  ATR  System.  The  dashed  line  under  Overall  Detection 
Performance  indicates  that  either  the  Defined  Pd  or  the  ROC  measures  are  employed.  The 
parenthetical  numbers  indicate  the  relative  weights  for  a  value,  within  the  parent  value. 


Parnell,  and  Meyers  (1999:  168).  The  relative  weight  deemed  most  important,  vv. ,  (generally  the 
largest  in  magnitude)  was  varied  by  a  factor  of  0.1,  or  w,+  =  l.lw,.  and  wj  =  0.9  w(. .  The 

remaining  weights  were  adjusted  to  meet  the  summation  constraint  while  retaining  their  relative 
ratios.  The  weights  then  become  a  single  variable  in  the  regression  to  find  the  response  surface. 

The  remaining  25  variables  (the  23  evaluation  measure  variables,  the  mission  profile,  and 
the  alternative  variables)  are  still  a  large  number  of  parameters.  To  reduce  the  number  to  a  more 
tractable  amount,  the  group  screening  technique  was  employed.  The  first  decomposition  of  the 
value  hierarchy,  the  second  row  from  the  top,  was  used  as  the  initial  group  screening  set.  RSM 
was  employed  on  this  groups  to  determine  significant  groups.  RSM  was  then  be  employed 
separately  for  each  decomposition  of  the  significant  value  groups.  The  evaluation  measure  least 
square  coefficients  from  the  RSM  analyses  were  then  combined  to  produce  a  single  list  of 
significant  evaluation  measures.  The  list  was  prioritized  based  on  least  squares  model 
coefficients.  This  prioritized  list  was  used  for  construction  of  the  hybrid  value-utility  model, 
which  was  then  used  to  analyze  the  decision  in  typical  fashion. 

5.2.1  Initial  Group  Screening 

For  the  ATR  analysis,  the  grouping  of  the  continuous  variables  was  selected  at  the  second 
level  of  the  value  hierarchy  (see  Figure  14).  This  provides  seven  groups: 

Robustness, 

Overall  Detection  Performance, 
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Employment  Concept, 

Declaration  Ability, 

Classification  Ability, 

Cost,  and 

Self-Assessed  Accuracy. 

Counting  the  weight  variable,  there  were  eight  continuous  variables.  There  are  also  two 
categorical  variables:  the  mission  profile,  with  two  levels,  CID  and  ISR;  and  the  alternative,  with 
three  levels,  ATR  33,  ATR  55,  and  ATR  89.  The  design  was  developed  in  the  JMP®  software 
package,  which  recommended  a  L36  Taguchi  Design.  Six  center  points  were  added  to  the  L36 
design,  permitting  the  various  combinations  of  profile  and  alternative  to  be  examined.  For  this 
example  only  first  order  effects,  without  interactions,  will  be  examined.  As  no  interactions  are 
under  consideration,  aliasing  concerns  are  avoided. 


ATR 


j  i  j 


Robustness 

Overall 

Employment 

Declaration 

Classification 

Cost 

Self  Assessed 

(0.2) 

Detetction 

Concept 

Ability 

Ability 

(0.10) 

Accuracy 

Performance 

[Rating] 

[Pdec] 

(0.11) 

(0.14) 

(0.17) 

(0.15) 

(0.13) 

Figure  14.  First  Group  Screening,  Second  Tier  of  Value  Hierarchy. 


The  L36  design  employs  the  continuous  variables  at  a  high  or  low  setting,  indicated  by  a 
plus  or  minus  sign,  respectively,  in  the  pattern  notation.  The  patterns  for  the  matrix  rows  are 
shown  in  Table  1.  For  the  continuous  variables,  the  minus  sign  indicates  the  variable  of  interest 
is  set  to  the  lower  level,  a  plus  sign  indicates  it  is  set  to  the  higher  level.  The  number  zero 
identifies  center  points.  For  the  categorical  variables,  which  are  handled  by  JMP®  with  dummy 
variables,  the  minus,  plus,  and  zero  indicate  different  categories.  For  example,  the  lead  minus 
sign  in  row  1  indicates  that  the  robustness  variable  is  at  the  lower  setting.  The  order  of  the 
variables  is  as  presented  in  the  list  above,  then  the  profile  and  alternative  variables.  The  data  for 
these  setting  are  presented  below  in  Equations  (50)  through.  Also  shown  in  Table  1  are  the 

results  for  each  combination  (row),  U  . 
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Table  1 .  Initial  Group  Screening  Design  and  Response  for  ATR  Problem. 
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mm 
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59 
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8 
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0.544 

22 
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-+—+++-+ 

0.414 
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23 

-+++-++-+0 

00000000- 

0.509 

10 
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0.601 

24 
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00000000-0 

0.556 

11 
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25 
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-++-+—++ 

0.647 

26 

--+++-++-0 

KITCK1 

40 

BhliMiltMiltlM 
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mm 

+-++-+- — 

0.574 

27 

-+++-++-+ 

0.541 

41 

00000000+0 

HESQHI 

mm 

0.622 

28 

— +++-++- 

0.381 

00000000++ 

WM3SSSMM 

To  determine  the  response,  the  decision  problem  was  modeled  in  the  DPL®  software 
package.  The  various  evaluation  measure  levels,  xi ,  were  passed  to  an  Excel®  spreadsheet 

where  v,  (xi )  was  calculated.  This  intermediate  result  was  used  to  determine  w(  (v(.  (x(. ))  from 


M.(v.  (*.))= 


1  -  e  /a 
\-eX//p‘ 


where 


Pi  = 


0. 1  for  pattern  "+" 
oo  for  pattern  "0" 
[-0.1  for  pattern 


A  zero  indicates  nominal  setting,  when  u{  (v,  (x, ))  =  v(  (x, ) . 


(49) 


(50) 


The  weight  variable  w  is  a  vector.  The  nominal  value  for  the  largest  weight  is  that  of  the 
Robustness  evaluation  measure  with  a  weight  of  0.2.  However  no  data  was  available  for  the 
robustness  for  identification  of  class.  Because  of  this,  the  next  largest  evaluation  measure  weight 
was  selected.  This  was  the  Overall  Detection  Performance  evaluation  measure,  denoted  w2 , 
equal  to  0.17.  Perturbing  this  by  0.1,  or  w2  =  w2  ±0.1w2 ,  and  adjusting  the  remaining  weights 
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produces  two  weight  vectors  as  called  for  by  the  high  and  low  pattern  indicators.  As  with  p{ ,  a 

zero  indicates  nominal  setting,  or  the  elicited  weights,  w° . 

These  vectors  are 

w+  for  pattern  "+" 

w°  for  pattern  "0"  (51) 

w"  for  pattern 


The  response  variable,  utility  U  ,  is  also  presented  in  Table  1.  A  plot  of  actual  versus 
predicted  values  is  contained  in  Figure  15.  The  linear  model  fit  well  with  R 2  =  0.978  and 
Kdj  =  0-970 .  The  P  value  was  less  than  0.0001 .  The  parameters  estimates  and  the  significance 

information  for  the  model  are  contained  in  Table  2.  The  Declaration  Ability  and  Self- 
Assessment  groups  are  clearly  not  significant.  They  may  be  dropped  from  consideration.  The 
weights  are  also  not  significant,  which  agrees  with  the  sensitivity  analysis  performed  as  part  of 
the  basic  DA  analysis  of  the  problem,  and  will  be  eliminated  from  further  consideration.  The 
other  groups  are  clearly  significant.  Note  that  the  coefficient  estimates  for  the  least  squares 
model  agree  with  the  significance  information.  The  significant  variables  have  coefficients  that 
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are  an  order  of  magnitude  larger  than  those  found  to  be  non-significant.  The  coefficients  also 
provide  a  measure  of  importance.  Repeating  the  model  fit  with  these  groups  removed  would 
provide  the  same  information  (although  the  p-values  would  change  slightly  because  of  changes 
in  the  number  of  degrees  of  freedom)  and  so  is  not  required. 


Figure  15.  ATR  Group  Screening  Model  Actual  versus  Predicted  Values. 


The  significance  information  in  Table  2  may  be  used  to  prioritize  the  order  of  converting 
the  single  dimensional  value  functions  to  their  utility  equivalents.  However  the  group  screening 
has  only  eliminated  five  evaluation  measures.  Further,  four  groups  have  their  significance  levels 
indicated  as  equal.  This  fails  to  prioritize  between  eight  evaluation  measures.  For  further 
refinement  the  next  step  is  to  decompose  the  groups,  to  determine  the  significance  of  the  group 
members. 

5.2.2  Second  Group  Screening. 

The  candidate  groups  for  decomposition  and  examination  with  RSM  are:  Robustness, 
Detection  Performance,  Classification  Ability,  and  Cost.  The  Employment  Concept  criterion  is 
not  decomposable,  so  cannot  be  examined  further.  Employment  Concept  should  be  converted  to 
utility  in  the  hybrid  model.  (However  of  note  is  that  all  three  candidates  have  equal  scores  for 
this  criterion.  If  no  additional  candidates  are  introduced  with  a  differing  score  for  Employment 
Concept,  and  no  sensitivity  analysis  is  performed  on  the  Employment  Concept  scores,  it  may  be 
retained  as  a  value  function.  Typically  the  sensitivity  analysis  restriction  is  onerous,  but  this 
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Table  2.  ATR  Group  Screening  Parameter  Estimates. 


Term 

Estimate 

t  Ratio 

Prob  >  |t| 

Intercept 

0.529 

0.00316 

167.50 

<.0001 

Robustness 

0.0647 

0.00341 

18.95 

<.0001 

Detection  Performance 

0.0577 

16.92 

<.0001 

Employment  Concept 

0.0738 

0.00341 

21.62 

<.0001 

Declaration  Ability 

0.00211 

0.00341 

0.62 

0.541 

Classification  Ability 

0.0456 

0.00341 

13.37 

<.0001 

Cost 

0.0139 

0.00341 

4.09 

Self  Assessment  Accuracy 

-0.00528 

0.00341 

-1.55 

-0.00556 

0.114 

CID  Profile 

0.0109 

0.00316 

3.46 

0.0016 

ISR  Profile 

.-0.0109 

0.00316 

-3.46 

0.0016 

ATR  33  Alternative 

-0.0157 

0.00447 

-3.52 

0.0014 

ATR  55  Alternative 

0.0104 

0.00447 

2.32 

0.0274 

ATR  89  Alternative 

0.00536 

0.00446 

1.20 

0.240 

point  might  be  valuable  where  many  utility  elicitations  are  involved.)  Eliminated  from  the 
immediate  requirement  to  convert  from  value  to  utility  functions  are  Declaration  Ability  and 
Self-Assessment.  Choosing  the  first  subgroup  to  screen  should  be  done  with  consideration  as  to 
the  potential  for  eliminating  criteria.  The  Cost  measure  provides  this,  but  is  complex  with  two 
sub-levels  and  eleven  evaluation  measures.  For  simplicity  Robustness  was  chosen;  other  groups 
were  addressed  later.  The  Robustness  portion  of  the  value  hierarchy  is  shown  in  Figure  16. 


Figure  16.  Second  Group  Screening,  Robustness  Portion  of  Value  Hierarchy. 
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Robustness  has  three  evaluation  measures:  Robustness  in  Detection  ( A Pd ),  Robustness  in 
Typification  ( A Pid ),  and  Robustness  in  Classification  ( A Pcc ).  The  variables  under  consideration 
are  the  exponential  constants  for  these  measures.  Refer  to  them  as  pvp2,  and  P^ »  respectively, 

for  this  portion  of  the  analysis.  They  take  on  values  as  indicated  in  equation  (50).  The  weights 
will  be  perturbed  as  above  with  the  largest  weight,  that  of  Robustness  in  Detection,  being 
perturbed  ten  percent  about  the  nominal  value.  The  equivalent  of  equation  (52)  becomes 

"0.46751  [0.425]  [0.3825" 

w+  =  0.2547  ,  w°  =  0.275  ,  w_  =  0.2953  (55) 

0.2778J  [  0.3  J  [o.3222_ 

The  profile  and  alternative  information  remained  unchanged.  Categorical  data  again 
restricted  the  possible  experimental  designs  available.  A  Hunter  L18  (Sail,  2001:  57)  design  was 
selected.  This  guarantees  orthogonality  and  there  are  no  aliasing  concerns,  as  we  do  not 
investigate  interactions  here.  The  Robustness  screening  experimental  design  is  illustrated  in 
Table  3,  along  with  the  results.  The  pattern  information  indicates  variables  levels  as  explained 
above,  with  a  variable  order  of  Robustness  in  Detection,  Robustness  in  Typification,  and 
Robustness  in  Classification,  weight,  profile,  and  alternative.  Analysis  of  these  data  provides  the 
coefficient  estimates  in  Table  4. 

As  can  be  seen  from  the  data  in  Table  4,  Robustness  in  Detection  and  Robustness  in 
Typification  are  significant  while  Robustness  in  Classification  is  not  significant.  The  first  two 
evaluation  measures  are  candidates  for  conversion  from  vy  (x}  )to  Uj  (xj )  while  the  last  is 

excluded  from  consideration.  The  mission  profile  is  not  significant.  Any  other  result  would  be 
surprising,  as  the  input  data  for  robustness  was  not  conditioned  on  mission  profile.  The 
alternatives  are  not  significant.  This  indicates  that  the  most  desired  alternative  changes  over  the 
range  of  the  perturbation  introduced  in  this  analysis. 

5.2.3  Third  Group  Screening. 

Turning  to  consider  cost,  analysis  proceeded  in  similar  fashion.  The  least  squares  fit  of 
the  data  is  provided  in  Table  5.  The  Cost  Use  Time  was  highly  significant,  but  the  model  fit  was 
mediocre,  with  R2  =  0.657  and  R2dj  =  0.523 .  Because  of  the  fit,  Cost  Redeployment  Time,  Cost 
Use  Experience,  and  Cost  Redeployment  Risk  were  retained  in  the  list  of  significant  variables. 
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Table  3.  Robustness  Screening  Experimental  Design  and  Response  for  ATR  Problem. 
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Row 

Pattern 
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8 

++-+0 
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16 
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Table  4.  ATR  Robustness  Screening  Parameter  Estimates. 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob  >  Itl 

Intercept 

0.628 

0.0127 

49.38 

<.0001 

Robustness  in  Detection 

0.226 

0.0149 

15.10 

<.0001 

Robustness  in  Typification 

0.140 

0.0149 

9.39 

<.0001 

Robustness  in  Classification 

0.0113 

0.0149 

0.75 

0.461 

Weights(-U) 

-0.0214 

0.0149 

-1.43 

0.171 

Profile[CID] 

0.0109 

0.0129 

0.84 

0.411 

Profile[ISR] 

-0.0109 

0.0129 

-0.84 

0.411 

Alternative  [ATR  33] 

-0.0207 

0.0180 

-1.15 

0.267 

Altemative[ATR  55] 

0.0213 

0.0180 

1.18 

0.254 

Alternative[ATR  89] 

-0.000597 

0.0180 

-0.03 

0.974 

Table  5.  ATR  Cost  Screening  Parameter  Estimates. 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob  >  Itl 

Intercept 

0.590 

0.0140 

42.27 

<.0001 

Development  Money(- 1,1) 

-0.00393 

0.0143 

-0.28 

0.785 

Development  Time(- 1,1) 

0.00171 

0.0143 

0.12 

0.906 

Development  Expertise(-l,l) 

0.0131 

0.0143 

0.91 

0.368 

Development  Risk(- 1,1) 

-0.00183 

0.0144 

-0.13 

0.899 

Redeployment  Money(- 1,1) 

0.0101 

0.0144 

0.70 

0.489 

Redeployment  Time(- 1,1) 

0.0239 

1.66 

Redeployment  Expertise!  - 1,1) 

0.0170 

0.244 

Redeployment  Risk(-l,l) 

0.0231 

| 

mm 

0.118 

Use  Money(-l,l) 

0.0169 

1.17 

0.249 

Use  Time(-l,l) 

HBQIiiSiJBSi 

0.0143 

3.97 

0.0003 

Use  Expertise!- 1.1) 

0.0144 

1.62 

0.115 

Weights!- 1,1) 

0.0261 

0.0144 

1.81 

0.0787 

ATR[ATR  33] 

0.0805 

0.0196 

4.10 

0.0002 

ATR[ATR  55] 

-0.114 

0.0196 

-5.83 

<.0001 

ATR  [ATR  89] 

0.0340 

0.0196 

1.73 

0.0924 

5.2.4  Fourth  Group  Screening 


The  Detection  Performance  taxon  was  next  examined.  This  decomposed  into  two 
evaluation  measures,  False  Alarm  Rate  given  a  Probability  of  Detection  and  the  Probability  of 
False  Alarm  given  a  Probability  of  Detection.  (Alternatively,  it  could  have  been  structured  on 
the  Required  Operating  Curve  False  Alarm  Rate  and  the  Required  Operating  Curve  Probability 
of  False  Alarm.)  Analysis  proceeded  as  above.  The  least  square  variable  significance  and 
parameter  estimates  are  in  Table  6.  As  seen  in  Table  6,  both  the  False  Alarm  Rate  and  the 
Probability  of  False  Alarm  were  found  to  be  significant. 


Table  6.  ATR  Detection  Performance  Screening  Parameter  Estimates. 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob  >  Itl 

0.471 

0.0247 

19.02 

FAR(-1,1) 

0.1293 

0.0290 

4.46 

| 

Pfa(-U) 

0.214 

0.0290 

7.38 

<.0001 

w(-l,l) 

-0.0207 

0.0290 

Profile  [CID] 

0.00384 

0.0250 

Profile[ISR] 

-0.00384 

0.0250 

0.880 

ATR[ATR  33] 

|HKgIGE9l| 

-5.58 

<.0001 

ATR[ATR  55] 

0.0653 

■ 

1.86 

0.0796 

ATR[ATR  89] 

0.130 

0.0350 

3.72 

0.0017 

5.2.5  Fifth  Group  Screening 

Next  Classification  Ability  was  analyzed.  The  coefficients  and  significance  statistics  are 
contained  in  Table  7.  Both  Classify  by  Type  and  Classify  by  Class  evaluation  measures  are 
highly  significant. 

5.2.6  Sixth  Group  Screening 

Finally  the  Self  Assessed  Accuracy  category  was  considered.  The  data  and  both  value 
and  utility  functions  are  identical  for  the  three  evaluation  measures  in  this  category.  The  input 
data  are  all  unity,  providing  value  and  utility  of  zero,  as  this  evaluation  measure  was  not  part  of 
the  ATR  CS  testing.  So  testing  this  category  as  above  was  useless,  as  all  combinations  of 
weights  and  utility  exponential  values  produce  a  utility  of  zero.  Clearly  the  Self  Assessed 
Accuracy  measures  are  not  significant. 


Table  7.  ATR  Classification  Ability  Screening  Parameter  Estimates. 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob  >  Itl 

Intercept 

0.534 

0.0204 

26.12 

<.0001 

Type(-l.l) 

0.215 

0.0240 

8.97 

<.0001 

Class(-l,l) 

0.267 

0.0239 

11.2 

<.0001 

w(-l,l)  1 

0.00705 

0.0239 

0.29 

0.772 

Profile  [CID] 

0.0649 

0.0207 

3.14 

0.0060 

Profile[ISR] 

-0.0649 

0.0207 

-3.14 

0.0060 

ATR[ATR  33] 

0.00477 

0.0289 

0.16 

0.871 

ATR[ATR  55] 

0.0129 

0.0289 

0.45 

0.661 

ATR[ATR  89] 

-0.0177 

0.0289 

-0.61 

0.550 

5.2.7  Prioritization  of  Evaluation  Measures 

The  ordered  coefficients  are  contained  in  Table  8.  They  were  determined  by  multiplying 
coefficients  by  the  coefficient  of  parent  groups  to  provide  a  global  comparison.  These  provide  a 
reasonable  order  in  which  single  dimensional  value  functions  should  be  converted  to  single 
dimensional  utility  functions  in  the  hybrid  model. 

The  value  hierarchy  reduced  to  those  evaluation  measures  that  were  determined  to  be 
significant  for  a  =  0.10  during  the  screening  process  is  depicted  in  Figure  17.  Examining  only 
these  evaluation  measures  produces  the  subset  of  model  coefficients  listed  in  Table  9.  The 
entries  of  Table  9  are  identical  to  the  top  eleven  entries  of  Table  8  except  that  Declaration  Ability 
and  Classification  Robustness  do  not  appear  in  Table  9.  These  evaluation  measures  were  not 
found  to  be  significant  during  the  screening.  They  are  in  the  eighth  and  tenth  positions  of  Table 
8.  This  indicates  that  employing  the  significance  as  a  screening  tool  is  reasonable. 


5.3  Employment  of  the  Hybrid  Value-Utility  Model 

The  hybrid  utility  algorithm  was  applied,  using  the  priority  sequence  of  variables 
presented  in  Table  9.  Iteration  zero  employs  the  value  model.  Each  succeeding  iteration 
provides  a  hybrid  model  with  the  next  highest  priority  evaluation  measure  value  function 
replaced  by  the  utility  function.  The  results  for  the  mission  CID  profile  are  presented  in  Table 
10.  The  pure  utility  model  results  are  also  provided. 
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Table  8.  Rank  Ordered,  by  Absolute  Value,  Model  Coefficients. 


Evaluation  Measure 

Coefficient 

Employment  Concept 

0.0738 

Detection  Robustness 

0.0146 

Probability  of  False  Alarm 

0.0123 

Classification  by  Class 

0.0122 

Classification  by  Type 

0.00980 

Identification  Robustness 

0.00907 

False  Alarm  Rate 

0.00746 

Declaration  Ability 

0.00211 

Cost  Use  Time 

0.000794 

Classification  Robustness 

0.000729 

Cost  Redeployment  Time 

0.000334 

Cost  Use  Expertise 

0.000324 

Cost  Redeployment  Risk 

0.000322 

Cost  Redeployment  Expertise 

0.000238 

Cost  Use  Money 

0.000235 

Cost  Development  Expertise 

0.000183 

Redeployment  Money 

0.000141 

Cost  Development  Money 

-5.48  x  10~5 

Cost  Development  Risk 

-2.55  x  1(T5 

Cost  Development  Time 

2.38  x  10~5 

Es  Pd 

0 

Es  Pid 

0 

Es  Pcc 

0 

Figure  17.  Value  Hierarchy  Reduced  to  Significant  Taxa.  The  parenthetical  numbers 
indicate  the  estimated  coefficients.  Taxa  significant  at  alpha  equal  to  0.1  and  not  when  alpha 

equal  0.05  are  indicated  with  an  asterisk. 


48 


Table  9.  Significant  Rank  Ordered  Model  Coefficients.  Those  significant  at  an  alpha  equal  to 
0.1  and  not  when  alpha  equal  0.05  are  indicated  with  an  asterisk. 


Evaluation  Measure 

Coefficient 

Priority 

Concept 

0.0738 

1 

Detection  Robustness 

0.0146 

2 

Probability  of  False  Alarm 

0.0123 

3 

Classification  by  Class 

0.0122 

4 

Classification  by  Type 

0.00980 

5 

Identification  Robustness 

0.00907 

6 

False  Alarm  Rate 

0.00746 

7 

Cost  Use  Time 

0.000794 

8 

Cost  Redeployment  Time* 

0.000334 

9 

Cost  Use  Expertise* 

0.000324 

10 

Cost  Redeployment  Risk* 

0.000322 

11 

The  hybrid  utility  model  results  are  shown  graphically  in  Figure  18  for  the  ATR  33 
alternative.  Other  alternatives  behaved  similarly  and  are  omitted  for  clarity.  All  three 
alternatives  converge  towards  the  utility  model  after  an  initial  period  of  instability.  This  is  more 
readily  observed  in  Figure  19  through  Figure  22,  where  the  differences  between  iterative  hybrid 
model  scores  and  the  distance  from  the  true  utility  model  for  the  two  mission  profiles  are 
illustrated.  The  decreasing  differences  between  iterations  and  the  decreasing  distance  from  the 
utility  model  are  obvious.  The  latter  measure  is  a  measure  of  the  error  of  the  hybrid  model 
compared  to  the  true  utility  model.  After  the  initial  period  this  error  remains  small. 

To  understand  the  behavior  of  the  hybrid  model  as  it  is  modified  during  each  iteration,  we 
examine  the  initial  step  for  the  ATR  33  alternative  (see  Table  10  and  Figure  18).  The  complete 
value  model  had  an  overall  value  of  0.509  for  this  alternative.  When  the  Employment  Concept 
was  converted  to  the  single  dimensional  utility  function,  the  hybrid  utility  became  0.464.  The 
Employment  Concept  rating  for  the  ATR  33  alternative  (indeed,  all  alternatives)  was  poorly 
defined.  The  corresponding  single  dimensional  value  is  0.4.  The  single  dimensional  utility  for 
the  poorly  defined  rating  is  0.1,  so  the  difference  between  the  preference  functions  is  0.3.  When 
this  difference  is  multiplied  by  the  relative  weight  for  the  Employment  Concept  evaluation 
measure,  0.15,  the  result  is  0.45,  the  difference  between  the  results  obtained  in  the  first  iteration 
compared  to  the  starting  value  model. 
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Table  10.  CID  Profile  Value,  Utility,  and  Hybrid  Utility  Results,  Significant  Evaluation 
Measures.  Those  significant  at  an  alpha  equal  to  0. 1  and  not  when  alpha  equal  0.05  are  indicated 

with  an  asterisk. 


Iteration 


Figure  19.  Hybrid  Utility  Model  Difference  Between  Successive  Iterations,  CID  Profile. 


Iteration 


Figure  20.  Hybrid  Utility  Model  Difference  Between  Successive  Iterations,  ISR  Profile. 
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Iteration 


Figure  21.  Hybrid  Utility  Model  Difference  Between  Iteration  and  True  Utility  Model,  CID 

Profile. 


Iteration 


Figure  22.  Hybrid  Utility  Model  Difference  Between  Iteration  and  True  Utility  Model,  ISR 

Profile. 
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It  is  interesting  to  note  in  Figure  18  that  ATR  33  converges  to  the  true  utility  model 
strictly  from  below  for  the  CID  profile,  although  the  direction  of  movement  was  not  identical  for 
each  iteration,  while  under  the  ISR  case  ATR  33  oscillated  about  the  true  utility  value.  In  no 
case  was  the  movement  strictly  towards  the  true  utility,  as  would  be  expected  if  the  decision 
maker  was  strictly  relatively  risk  averse  (seeking). 

Once  the  significant  evaluation  measures,  at  a  -  0.05 ,  were  converted  from  value  to 
utility  functions  the  hybrid  function  provided  estimates  that  differed  at  most  0.003  from  the  pure 
utility  function  result.  The  evaluation  measures  that  became  significant  between  a  -  0.05  and 
a  =  0. 1  provided  no  improved  estimate. 

We  conclude  that  for  the  example  the  evaluation  measures  determined  to  be  significant 
through  response  surface  methodology  techniques  provided  an  adequate  set  of  variables  to 
permit  creation  of  an  accurate  hybrid  model.  The  hybrid  model  may  be  used  to  represent  the 
utility  model.  The  number  of  preference  functions  required  to  be  elicited  from  the  decision 
maker  was  reduced  from  26  to  eight  for  a  =  0.05 . 

Further,  the  rank  ordering  by  regression  coefficient  was  successful  in  providing 
successive  hybrid  estimates  that  in  general  approached  the  true  utility  value,  each  with  generally 
reduced  difference  {8i )  from  the  previous  estimate. 
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6  Summary 


A 

The  algorithm  provides  a  hybrid  value-utility  model.  The  hybrid  utility  model,  U  , 
provided  an  adequate  representation  of  the  true  utility  model,  U  .  Elicitation  of  the  hybrid  utility 
model  is  more  efficient  with  respect  to  decision  maker  participation  than  employing  the  strict 
utility  model,  especially  in  cases  where  there  are  a  large  number  of  evaluation  measures  or  cases 
where  several  of  the  measures  require  an  inordinate  amount  of  time  to  elicit  additional  utility 
information.  Such  efficiency  promotes  use  of  decision  analysis  and  encourages  sound  decision¬ 
making.  Under  conditions  of  uncertainty  and  risk,  the  value  model  may  provide  differing  results 
than  the  utility  model,  as  was  demonstrated  in  the  example,  and  so  is  not  acceptable. 

This  algorithm  is  unique  in  several  ways.  Response  surface  methodology  is  used  to 
determine  when  the  value  model  is  sensitive  to  the  preference  functions.  Response  surface 
methodology  was  used  to  examine  the  margins  of  an  envelope  of  reasonable  curves  for 

u.  (v,.  (jq  )) .  When  the  function  failed  to  be  significant,  then  the  true  function  was  assumed  to  be 
not  significant  for  evaluation  measure  i. 

While  value  and  utility  are  differing  constructs,  prudent  exploitation  of  their  differences 
and  similarities  permits  more  efficient  use  of  the  decision  maker’s  time.  This  approach 
emphasizes  that  the  distinction  between  value  and  utility  must  not  be  ignored.  But  the  lack  of 
strategic  differences  for  specific  cases  may  be  identified  and  exploited 


54 


Appendix  A:  Von  Neumann  Expected  Utility  Axioms 


Others  have  rewritten  the  von  Neumann  and  Morgenstern  (1953)  axioms  for  clarity.  For 
their  description,  the  presentation  generally  follows  Biswas  (1997:  17  -  18).  Let  E  denote  the 
outcome  set,  so  x  e  E  is  an  outcome.  Let  X  =  (x, ,  x2 ,  •  •  • ,  xn )  be  the  vector  of  n  possible  discrete 

realizations  from  the  outcome  set,  each  with  a  respective  probability  of  occurrence,  pl . 
Represent  a  lottery  of  these  n  outcomes  with  L  -  (p,x, ,  p2x2 ,  •  ••,pnxn ).  Preference  relationships 
will  be  written  for  x,.,x .  e  X  as  x,.  >-  xj  when  xi  is  preferred  to  Xj ,  and  xt  ~  x;.  when  a  subject 
is  indifferent  between  x  and  Xj .  Similarly,  to  represent  x;  is  at  least  as  preferred  as  Xj ,  we 
write  x(.  >3  x} .  Without  loss  of  generality,  we  will  assume  that  x,  >3  x2  >3  •  •  •  >3  xn . 

A.l  Axiom  1  (Ordering). 

Preferences  may  be  (totally)  ordered  between  any  two  outcomes.  This  ordering  is  a  total 
ordering.  So  for  any  (x(.,x,.)  either  xf  >3  x}  or  x}  >3  xr  Further,  x;  >3  xj  and  Xj  >3  xk  implies 

Xi£xj£xk- 

A.2  Axiom  2  (Reduction  of  Compound  Lotteries). 

An  individual  is  indifferent  between  a  simple  lottery  L,  (a  lottery  with  outcomes  that  are 
fixed)  and  a  compound  lottery  (a  lottery  with  one  or  more  outcomes  that  are  also  lotteries)  with 
the  same  outcomes  when  the  probabilities  of  the  simple  lottery  are  determined  in  accordance 
with  standard  manipulation  of  probabilities.  If  a  lottery  L ^  is  defined  as 
L(m)  =  {p"' x, ,  p2x2 ,  •  •  • ,  p™xn ),  where  the  superscripts  indicate  membership  in  a  compound 

lottery  m,  it  follows  that  ((y,L^,^2L(2V •  •  ,q0&^\ ~  (p,x1,p2x2,---,p„xn)  where 
Pi  =  qtpP  +  — I -q„p\°^  and  ~  represents  indifference. 

For  clarity,  this  axiom  will  be  detailed  for  the  case  of  two  lotteries  each  with  possible 
outcomes  x,  and  x2 .  These  lotteries  are  L(l)  =  (p,^x, ,  p2^x2 )  and  iS1'1  =  (p^x, ,  p(f>x2 ).  They 
are  shown  graphically  in  Figure  23.  Then  (p,x1,p2x2,---,pnxn) 
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becomes  (qiL{l),q2L{2>)~  {pxx{,p2x2)  where  pi  -qtpf  +  ---  +  q0p\o)  or  pl  =  qxp\x)  +  q2p\1]  and 
P2  =  QiPV  +  (hP? ■  So  an  individual  would  be  indifferent  between  the  choice  of  the  compound 
lottery  or  the  corresponding  outcomes  with  the  associated  reduced  probabilities, 
{q^],q2L(2))~{pxxx,p2x2). 


Lottery  Lottery  Lottery  L 


Figure  23.  Lotteries  La>  L(2\  and  L. 


A.3  Axiom  3  (Continuity). 

An  individual  will  be  indifferent  between  some  xt  and  a  lottery  with  outcomes  xt  and 
xn .  That  is,  there  exists  some  probability  qt  such  that  x.  ~  (qix,,0x2 ,0x2, •••,(! -#,)*„).  Define 

this  lottery  Lx~  =  (<?,*, , (1  -q,)xn). 

A.4  Axiom  4  (Substitutability). 

In  any  lottery,  Lx‘  may  be  substituted  for  x( . 

A.5  Axiom  5  (Transitive  Preference  over  Lotteries). 

Preference  ordering  is  transitive  over  lotteries.  That  is  zl'^  >3  zP  and  Zp^  £3  zP^  implies 


A.6  Axiom  6  (Monotonicity). 

An  individual  prefers  a  lottery  L  =  (px,,(l  -  p)x2)  to  a  distinct  lottery 
L'  =  ( p'x , ,  (1  -  p')x2 )  if  and  only  if  p>  p  .  Similarly,  an  individual  is  indifferent  to  a  lottery 
L  =  (px, ,  (1  -  p )x2 )  when  compared  to  another  lottery  L'  =  ( p\ ,  (1  -  p')x2 )  if  and  only  if 

p  =  p. 
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Appendix  B:  Measurable  Value  Function  Axioms 


Krantz,  Luce,  Suppes,  and  Tversky  (1971)  provide  a  set  of  axioms  that  establish  when 
relationships  between  objects  may  be  defined  as  an  algebraic  difference  structure.  (Other 
sources  of  the  axioms,  or  alternative  formulations  of  them,  are  [Fishburn,  1970],  [Suppes  and 
Winet,  1955],  and  [Debrau,  I960].)  When  these  axioms  hold,  a  real-valued  function  exists  that 
provides  cardinal  preference  information  about  the  objects. 

B.l  Definition  (Algebraic  Difference  Structure). 

The  notation  ab  represents  the  increase  in  preference  resultant  from  receiving  object  b  in  lieu  of 
object  a .  Suppose  A  is  a  nonempty  set  and  y  is  a  binary  relation  on  Ax  A.  The  pair 
(Ax  A,>:)is  an  algebraic  difference  structure  if  and  only  if  for  all  a,b,c,d,e,f  e  A  and  all 
sequences  al ,  a2 ,  •  •  • ,  a( , ,  •  •  •  e  A ,  the  following  axioms  hold  true: 

B.2  Axiom  1. 

(AxA,fc)  is  of  weak  order.  A  relation,  R ,  on  a  set,  A ,  is  of  weak  order  when  it  is 

transitive  and  strongly  complete.  Transitivity  exists  when  aRb  and  bRc  implies  aRc  for  all 
a,b,ce  A.  A  relation  is  strongly  complete  on  a  set  when  aRb  or  bRa  holds  for  all  a, be  A, 
including  when  a  =  b .  (Roberts,  1979:  20  and  29) 

B.3  Axiom  2. 

If  ab^cd,  then  dc^ba. 

B.4  Axiom  3. 

If  ab  y  de  and  bc'^ef  ,  then  ac  y  df  . 
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B.5  Axiom  4. 

If  ab>zcd>zaa,  then  there  exist  e,fe  A  ,  such  that  ae~cd  ~  fb.  (The  requirement  that 
ab  >z  cdbt  £  aa  ensures  that  ab  is  increasing.  This  is  required  so  that  e,f  be  members  of  the 

interval  [a,b] .  When  e,feab  then  it  must  be  true  that  e,f  e  A.) 

B.6  Axiom  5. 

If  a, ,  a2 ,  •  •  • ,  a, ,  •  •  •  is  a  strictly  bounded  standard  sequence  aMat  ~  a2a[  for  every  ai ,  aM  in 
the  sequence,  not  a2a,  ~  a,  a, ;  and  there  exists  e,fe  A  such  that  ef  >-  a, a,  >-  fe  for  all  a,  in  the 
sequence),  then  it  is  finite.  (Krantz,  Luce,  Suppes,  and  Tversky,  1971:  151) 

Krantz  et  al  show  that  if  (AxA,^)  is  an  algebraic  difference  structure,  then  there  exists  a 
real-valued  function  v  on  A  such  that,  for  all  a,b,c,de  A,  ab £3  cd  iff 

v(a)-v(b)>  v(c)-v(d)  and  v  is  unique  up  to  a  linear  transformation.  Methods  of  elicitation 

of  measurable  value  functions  are  described  by  Fishburn  (1967  and  1976);  Johnson  and  Huber 
(1977);  Kneppereth,  Gustafson,  Leifer,  and  Johnson  (1974);  Dyer  and  Sarin,  (1979  and  1982); 
Farquhar  and  Keller,  (1989);  and  Camacho  (1980). 
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Appendix  C:  List  of  Abbreviations 


A 

ATR 

Automatic  Target  Recognition 

C 

CID 

Combat  Identification 

CS 

Classification  Systems 

D 

DA 

Decision  Analysis 

DSA 

Dependent  sensitivity  analysis 

I 

ISR 

Intelligence/Surveillance/Reconnaissance 

O 

ORCEN 

Operations  Research  Center 

R 

ROSA 

Rank  Order  Stability  Analysis 

RSM 

Response  Surface  Methodology 
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